ABSTRACT
We introduce an optimal control method for electromagnetic haptic guidance systems. Our real-time approach assists users in pen-based tasks such as drawing, sketching or designing. The key to our control method is that it guides users, yet does not take away agency. Existing approaches force the stylus to a continuously advancing setpoint on a target trajectory, leading to undesirable behavior such as loss of haptic guidance or unintended snapping. Our control approach, in contrast, gently pulls users towards the target trajectory, allowing them to always easily override the system to adapt their input spontaneously and draw at their own speed. To achieve this flexible guidance, our optimization iteratively predicts the motion of an input device such as a pen, and adjusts the position and strength of an underlying dynamic electromagnetic actuator accordingly. To enable real-time computation, we additionally introduce a novel and fast approximate model of an electromagnet. We demonstrate the applicability of our approach by implementing it on a prototypical hardware platform based on an electromagnet moving on a bi-axial linear stage, as well as a set of applications. Experimental results show that our approach is more accurate and preferred by users compared to open-loop and time-dependent closed-loop approaches.
Supplemental Material
- David A. Abbink, Mark Mulder, and Erwin R. Boer. 2012. Haptic shared control: smoothly shifting control authority? Cognition, Technology & Work 14, 1 (2012), 19--28. DOI: http://dx.doi.org/10.1007/s10111-011-0192--5Google ScholarDigital Library
- A. Pedro Aguiar, Joao P. Hespanha, and Petar V. Kokotovic. 2008. Performance limitations in reference tracking and path following for nonlinear systems. Automatica 44, 3 (2008), 598 -- 610. DOI: http://dx.doi.org/10.1016/j.automatica.2007.06.030Google ScholarDigital Library
- Emre Aksan, Fabrizio Pece, and Otmar Hilliges. 2018. DeepWriting: Making Digital Ink Editable via Deep Generative Modeling. In Proceedings of the 2018 CHI Conference on Human Factors in Computing Systems (CHI '18). Association for Computing Machinery, New York, NY, USA, 1--14. DOI: http://dx.doi.org/10.1145/3173574.3173779Google ScholarDigital Library
- Karl Johan Åström and Tore Hägglund. 1995. PID controllers: theory, design, and tuning. Vol. 2. Instrument society of America Research Triangle Park, NC.Google Scholar
- COMSOL Multiphysics. COMSOL Multiphysics. (????). www.comsol.com.Google Scholar
- M. Da Silva, Y. Abe, and J. Popovic.´ 2008. Simulation of Human Motion Data using Short-Horizon Model-Predictive Control. Computer Graphics Forum 27, 2 (2008), 371--380. DOI: http://dx.doi.org/10.1111/j.1467--8659.2008.01134.xGoogle ScholarCross Ref
- Alexander Domahidi and Juan Jerez. 2014. FORCES Professional. embotech GmbH (http://embotech. com/FORCES-Pro). (2014).Google Scholar
- T. Faulwasser, B. Kern, and R. Findeisen. 2009. Model predictive path-following for constrained nonlinear systems. In Proceedings of the 48h IEEE Conference on Decision and Control (CDC). 8642--8647. DOI: http://dx.doi.org/10.1109/CDC.2009.5399744Google Scholar
- Piyum Fernando, Roshan Lalintha Peiris, and Suranga Nanayakkara. 2014. I-Draw: Towards a Freehand Drawing Assistant. In Proceedings of the 26th Australian Computer-Human Interaction Conference on Designing Futures: The Future of Design (OzCHI '14). Association for Computing Machinery, New York, NY, USA, 208--211. DOI: http://dx.doi.org/10.1145/2686612.2686644Google ScholarDigital Library
- Benjamin A. C. Forsyth and Karon E. MacLean. 2006. Predictive Haptic Guidance: Intelligent User Assistance for the Control of Dynamic Tasks. IEEE Transactions on Visualization and Computer Graphics 12, 1 (Jan. 2006), 103--113. DOI:http://dx.doi.org/10.1109/TVCG.2006.11Google ScholarDigital Library
- Christoph Gebhardt,, and Otmar Hilliges. 2018. Optimizing for Aesthetically Pleasing Quadrotor Camera Motion. ACM Trans. Graph. 37, 4, Article 90 (July 2018), 11 pages. DOI: http://dx.doi.org/10.1145/3197517.3201390Google ScholarDigital Library
- Bruce P Gibbs. 2011. Advanced Kalman fltering, least-squares and modeling: a practical handbook. John Wiley & Sons.Google Scholar
- Soheil Kianzad, Yuxiang Huang, Robert Xiao, and Karon E. MacLean. 2020. Phasking on Paper: Accessing a Continuum of PHysically Assisted SKetchING. In Proceedings of the 2020 CHI Conference on Human Factors in Computing Systems (CHI '20). Association for Computing Machinery, New York, NY, USA, 1--12. DOI:http://dx.doi.org/10.1145/3313831.3376134Google Scholar
- Hyunjung Kim, Seoktae Kim, Boram Lee, Jinhee Pak, Minjung Sohn, Geehyuk Lee, and Woohun Lee. 2008. Digital Rubbing: Playful and Intuitive Interaction Technique for Transferring a Graphic Image onto Paper with Pen-Based Computing. In CHI '08 Extended Abstracts on Human Factors in Computing Systems (CHI EA '08). Association for Computing Machinery, New York, NY, USA, 2337--2342. DOI: http://dx.doi.org/10.1145/1358628.1358680Google ScholarDigital Library
- Andrey Kolmogorov. 1933. Sulla determinazione empirica di una lgge di distribuzione. Inst. Ital. Attuari, Giorn. 4 (1933), 83--91.Google Scholar
- Denise Lam, Chris Manzie, and Malcolm Good. 2010. Model predictive contouring control. In Decision and Control (CDC), 2010 49th IEEE Conference on. IEEE, 6137--6142. DOI: http://dx.doi.org/10.1109/CDC.2010.5717042Google ScholarCross Ref
- Denise Lam, Chris Manzie, and Malcolm C Good. 2013. Model predictive contouring control for biaxial systems. IEEE Transactions on Control Systems Technology 21, 2 (2013), 552--559. DOI: http://dx.doi.org/10.1109/TCST.2012.2186299Google ScholarCross Ref
- Thomas Langerak, Juan Zarate, David Lindlbauer, Christian Holz, and Otmar Hilliges. 2020. Omni: Volumetric Sensing and Actuation of Passive Magnetic Tools for Dynamic Haptic Feedback. In Proceedings of the Annual Symposium on User Interface Software and Technology (UIST '20).Google ScholarDigital Library
- Long-Fei Lin, Shan-Yuan Teng, Rong-Hao Liang, and Bing-Yu Chen. 2016. Stylus Assistant: Designing Dynamic Constraints for Facilitating Stylus Inputs on Portable Displays. In SIGGRAPH ASIA 2016 Emerging Technologies (SA '16). Association for Computing Machinery, New York, NY, USA, Article Article 14, 2 pages. DOI:http://dx.doi.org/10.1145/2988240.2988255Google ScholarDigital Library
- Alexander Liniger, Alexander Domahidi, and Manfred Morari. 2014. Optimization-based autonomous racing of 1:43 scale RC cars. Optimal Control Applications and Methods (2014). DOI: http://dx.doi.org/10.1002/oca.2123Google Scholar
- Pedro Lopes, Do?aa Yüksel, François Guimbretière, and Patrick Baudisch. 2016. Muscle-Plotter: An Interactive System Based on Electrical Muscle Stimulation That Produces Spatial Output. In Proceedings of the 29th Annual Symposium on User Interface Software and Technology (UIST '16). Association for Computing Machinery, New York, NY, USA, 207--217. DOI: http://dx.doi.org/10.1145/2984511.2984530Google ScholarDigital Library
- Jess McIntosh, Paul Strohmeier, Jarrod Knibbe, Sebastian Boring, and Kasper Hornbæk. 2019. Magnetips: Combining Fingertip Tracking and Haptic Feedback for Around-Device Interaction. In Proceedings of the 2019 CHI Conference on Human Factors in Computing Systems (CHI '19). Association for Computing Machinery, New York, NY, USA, 1--12. DOI:http://dx.doi.org/10.1145/3290605.3300638Google ScholarDigital Library
- morph. 2017. Sensel Morph Official Site. (2017). https://sensel.com/pages/the-sensel-morph.Google Scholar
- M.W. Mueller and R. D'Andrea. 2013. A model predictive controller for quadrocopter state interception. In Proceedings of the European Control Conference (ECC), 2013. 1383--1389. DOI: http://dx.doi.org/10.23919/ECC.2013.6669415Google ScholarCross Ref
- James Mullins, Christopher Mawson, and Saeid Nahavandi. 2005. Haptic handwriting aid for training and rehabilitation. In Systems, Man and Cybernetics, 2005 IEEE International Conference on, Vol. 3. IEEE, 2690--2694. DOI: http://dx.doi.org/10.1109/ICSMC.2005.1571556Google ScholarCross Ref
- Tobias Nägeli, Lukas Meier, Alexander Domahidi, Javier Alonso-Mora, and Otmar Hilliges. 2017. Real-time Planning for Automated Multi-View Drone Cinematography. ACM Transactions on Graphics (Proceedings of ACM SIGGRAPH) (2017).Google Scholar
- Ken Nakagaki and Yasuaki Kakehi. 2014. Comp*Pass: A Compass-Based Drawing Interface. In CHI '14 Extended Abstracts on Human Factors in Computing Systems (CHI EA '14). Association for Computing Machinery, New York, NY, USA, 447--450. DOI: http://dx.doi.org/10.1145/2559206.2574766Google ScholarDigital Library
- Gian Pangaro, Dan Maynes-Aminzade, and Hiroshi Ishii. 2002. The Actuated Workbench: Computer-Controlled Actuation in Tabletop Tangible Interfaces. In Proceedings of the 15th Annual ACM Symposium on User Interface Software and Technology (UIST '02). Association for Computing Machinery, New York, NY, USA, 181--190. DOI: http://dx.doi.org/10.1145/571985.572011Google ScholarDigital Library
- Huaishu Peng, Amit Zoran, and François V. Guimbretière. 2015. D-Coil: A Hands-on Approach to Digital 3D Models Design. In Proceedings of the 33rd Annual ACM Conference on Human Factors in Computing Systems (CHI '15). Association for Computing Machinery, New York, NY, USA, 1807--1815. DOI: http://dx.doi.org/10.1145/2702123.2702381Google Scholar
- R Tyrrell Rockafellar and Roger J-B Wets. 2009. Variational analysis. Vol. 317. Springer Science & Business Media.Google Scholar
- Roy Shilkrot, Pattie Maes, and Amit Zoran. 2014. Physical Rendering with a Digital Airbrush. In ACM SIGGRAPH 2014 Studio (SIGGRAPH '14). Association for Computing Machinery, New York, NY, USA, Article Article 40, 1 pages. DOI: http://dx.doi.org/10.1145/2619195.2656328Google ScholarDigital Library
- Edgar Simo-Serra, Satoshi Iizuka, and Hiroshi Ishikawa. 2018. Mastering Sketching: Adversarial Augmentation for Structured Prediction. ACM Trans. Graph. 37, 1, Article 11 (Jan. 2018), 13 pages. DOI: http://dx.doi.org/10.1145/3132703Google ScholarDigital Library
- Edgar Simo-Serra, Satoshi Iizuka, Kazuma Sasaki, and Hiroshi Ishikawa. 2016. Learning to Simplify: Fully Convolutional Networks for Rough Sketch Cleanup. ACM Trans. Graph. 35, 4, Article 121 (July 2016), 11 pages. DOI:http://dx.doi.org/10.1145/2897824.2925972Google ScholarDigital Library
- Evan Strasnick, Jackie Yang, Kesler Tanner, Alex Olwal, and Sean Follmer. 2017. ShiftIO: Reconfgurable Tactile Elements for Dynamic Affordances and Mobile Interaction. In Proceedings of the 2017 CHI Conference on Human Factors in Computing Systems (CHI '17). Association for Computing Machinery, New York, NY, USA, 5075--5086. DOI: http://dx.doi.org/10.1145/3025453.3025988Google ScholarDigital Library
- Ryo Suzuki, Jun Kato, Mark D. Gross, and Tom Yeh. 2018. Reactile: Programming Swarm User Interfaces through Direct Physical Manipulation. In Proceedings of the 2018 CHI Conference on Human Factors in Computing Systems (CHI '18). Association for Computing Machinery, New York, NY, USA, 1--13. DOI: http://dx.doi.org/10.1145/3173574.3173773Google ScholarDigital Library
- Akiko Teranishi, Georgios Korres, Wanjoo Park, and Mohamad Eid. 2018. Combining full and partial haptic guidance improves handwriting skills development. IEEE transactions on haptics 11, 4 (2018), 509--517. DOI:http://dx.doi.org/10.1109/TOH.2018.2851511Google ScholarDigital Library
- Malte Weiss, Chat Wacharamanotham, Simon Voelker, and Jan Borchers. 2011. FingerFlux: Near-Surface Haptic Feedback on Tabletops. In Proceedings of the 24th Annual ACM Symposium on User Interface Software and Technology (UIST '11). Association for Computing Machinery, New York, NY, USA, 615--620. DOI:http://dx.doi.org/10.1145/2047196.2047277Google ScholarDigital Library
- Jun Xing, Li-Yi Wei, Takaaki Shiratori, and Koji Yatani. 2015. Autocomplete Hand-Drawn Animations. ACM Trans. Graph. 34, 6, Article 169 (Oct. 2015), 11 pages. DOI:http://dx.doi.org/10.1145/2816795.2818079Google ScholarDigital Library
- Junichi Yamaoka and Yasuaki Kakehi. 2013. DePENd: Augmented Handwriting System Using Ferromagnetism of a Ballpoint Pen. In Proceedings of the 26th Annual ACM Symposium on User Interface Software and Technology (UIST '13). Association for Computing Machinery, New York, NY, USA, 203--210. DOI: http://dx.doi.org/10.1145/2501988.2502017Google ScholarDigital Library
- Shunsuke Yoshida, Haruo Noma, and Kenichi Hosaka. 2006. Proactive desk II: Development of a new multi-object haptic display using a linear induction motor. In Virtual Reality Conference, 2006. IEEE, 269--272. DOI:http://dx.doi.org/10.1109/VR.2006.110Google ScholarDigital Library
- Kar W Yung, Peter B Landecker, and Daniel D Villani. 1998. An analytic solution for the force between two magnetic dipoles. Physical Separation in Science and Engineering 9, 1 (1998), 39--52.Google Scholar
- Juan Zarate, Thomas Langerak, Bernhard Thomaszewski, and Otmar Hilliges. 2020. Contact-free Nonplanar Haptics with a Spherical Electromagnet. In 2020 IEEE Haptics Symposium (HAPTICS). 698--704.Google ScholarCross Ref
- Amit Zoran and Joseph A. Paradiso. 2013. FreeD: A Freehand Digital Sculpting Tool. In Proceedings of the SIGCHI Conference on Human Factors in Computing Systems (CHI '13). Association for Computing Machinery, New York, NY, USA, 2613--2616. DOI: http://dx.doi.org/10.1145/2470654.2481361Google ScholarDigital Library
- Amit Zoran, Roy Shilkrot, Pragun Goyal, Pattie Maes, and Joseph A Paradiso. 2014. The wise chisel: The rise of the smart handheld tool. IEEE Pervasive Computing 13, 3 (2014), 48--57. DOI: http://dx.doi.org/10.1109/MPRV.2014.59.Google ScholarCross Ref
Index Terms
- Optimal Control for Electromagnetic Haptic Guidance Systems
Recommendations
Optimal and nonlinear decoupling control of systems with sandwiched backlash
A scheme is proposed for control of multi-body, multi-input and multi-output nonlinear systems with joint backlash, flexibility and damping, represented by a gun turret-barrel model which consists of two subsystems: two motors driving two loads (turret ...
A new haptic device for applications in virtual reality and humanoid robotics
Informatics in Control, Automation and RoboticsThis paper presents an innovative haptic interface that allows to simulate a generic touch sensation thanks to the integration of an electro-cutaneous device and a force-feedback system. After investigating different levels of interaction with the human ...
Quasi-Optimal Robust Stabilization of Control Systems
In this paper, we investigate the problem of semiglobal minimal time robust stabilization of analytic control systems with controls entering linearly, by means of a hybrid state feedback law. It is shown that in the absence of minimal time singular ...
Comments