Huy Viet Le
Sarah Clinch
Corina Sas
Tilman Dingler
Niels Henze
Nigel Davies
ABSTRACT
Reviewing lifelogging data has been proposed as a useful tool to support human memory. However, the sheer volume of data (particularly images) that can be captured by modern lifelogging systems makes the selection and presentation of material for review a challenging task. We present the results of a five-week user study involving 16 participants and over 69,000 images that explores both individual requirements for video summaries and the differences in cognitive load, user experience, memory experience, and recall experience between review using video summarisations and non-summary review techniques. Our results can be used to inform the design of future lifelogging data summarisation systems for memory augmentation.
- Omid Aghazadeh, Josephine Sullivan, and Stefan Carlsson. 2011. Novelty detection from an ego-centric perspective. In Computer Vision and Pattern Recognition (CVPR), 2011 IEEE Conference on. IEEE, 3297-3304. DOI: http://dx.doi.org/10.1109/CVPR.2011.5995731 Google Scholar
Digital Library
- Emma Berry, Narinder Kapur, Lyndsay Williams, Steve Hodges, Peter Watson, Gavin Smyth, James Srinivasan, Reg Smith, Barbara Wilson, and Ken Wood. 2007. The use of a wearable camera, SenseCam, as a pictorial diary to improve autobiographical memory in a patient with limbic encephalitis: A preliminary report. Neuropsychological Rehabilitation 17, 4--5 (2007), 582-601. DOI: http://dx.doi.org/10.1080/09602010601029780Google ScholarCross Ref
- John Boreczky, Andreas Girgensohn, Gene Golovchinsky, and Shingo Uchihashi. 2000. An Interactive Comic Book Presentation for Exploring Video. In Proceedings of the SIGCHI Conference on Human Factors in Computing Systems (CHI '00). ACM, NY, NY, USA, 185-192. DOI: http://dx.doi.org/10.1145/332040.332428 Google ScholarDigital Library
- GAna Browne, Emma Berry, Narinder Kapur, Steve Hodges, Gavin Smyth, Peter Watson, and Ken Wood. 2011. SenseCam improves memory for recent events and quality of life in a patient with memory retrieval difficulties. Memory 19, 7 (2011), 713-722. DOI: http://dx.doi.org/10.1080/09658211.2011.614622Google ScholarCross Ref
- Janko Calic, David P Gibson, and Neill W Campbell. 2007. Efficient layout of comic-like video summaries. Circuits and Systems for Video Technology, IEEE Transactions on 17, 7 (2007), 931-936. DOI: http://dx.doi.org/10.1109/TCSVT.2007.897466 Google ScholarDigital Library
- Yi Chen and Gareth J. F. Jones. 2010. Augmenting Human Memory Using Personal Lifelogs. In Proceedings of the 1st Augmented Human International Conference (AH '10). ACM, NY, NY, USA, Article 24, 9 pages. DOI: http://dx.doi.org/10.1145/1785455.1785479 Google ScholarDigital Library
- Kai-Yin Cheng, Sheng-Jie Luo, Bing-Yu Chen, and Hao-Hua Chu. 2009. SmartPlayer: User-centric Video Fast-forwarding. In Proceedings of the SIGCHI Conference on Human Factors in Computing Systems (CHI '09). ACM, NY, NY, USA, 789-798. DOI: http://dx.doi.org/10.1145/1518701.1518823 Google ScholarDigital Library
- Patrick Chiu, Andreas Girgensohn, and Qiong Liu. 2004. Stained-glass visualization for highly condensed video summaries. In Multimedia and Expo, 2004. ICME'04. 2004 IEEE International Conference on, Vol. 3. IEEE, 2059-2062. DOI: http://dx.doi.org/10.1109/ICME.2004.1394670Google ScholarCross Ref
- Sarah Clinch, Nigel Davies, Mateusz Mikusz, Paul Metzger, Marc Langheinrich, Albrecht Schmidt, and Geoff Ward. 2016. Collecting Shared Experiences through Lifelogging: Lessons Learned. Pervasive Computing, IEEE 15, 1 (Jan 2016). Google ScholarDigital Library
- Sarah Clinch, Paul Metzger, and Nigel Davies. 2014. Lifelogging for 'Observer' View Memories: An Infrastructure Approach. In Proceedings of the 2014 ACM International Joint Conference on Pervasive and Ubiquitous Computing: Adjunct Publication (UbiComp '14 Adjunct). ACM, NY, NY, USA, 1397-1404. DOI: http://dx.doi.org/10.1145/2638728.2641721 Google ScholarDigital Library
- Matthew L. Cooper. 2011. Clustering Geo-tagged Photo Collections Using Dynamic Programming. In Proceedings of the 19th ACM International Conference on Multimedia (MM '11). ACM, NY, NY, USA, 1025-1028. DOI: http://dx.doi.org/10.1145/2072298.2071929 Google ScholarDigital Library
- Fergus I Craik and Joan M McDowd. 1987. Age differences in recall and recognition. Journal of Experimental Psychology: Learning, Memory, and Cognition 13, 3 (1987), 474.Google ScholarCross Ref
- N. Davies, A. Friday, S. Clinch, C. Sas, M. Langheinrich, G. Ward, and A. Schmidt. 2015. Security and Privacy Implications of Pervasive Memory Augmentation. Pervasive Computing, IEEE 14, 1 (Jan 2015), 44-53. DOI: http://dx.doi.org/10.1109/MPRV.2015.13Google Scholar
- Aiden R. Doherty, Daragh Byrne, Alan F. Smeaton, Gareth J.F. Jones, and Mark Hughes. 2008. Investigating Keyframe Selection Methods in the Novel Domain of Passively Captured Visual Lifelogs. In Proceedings of the 2008 International Conference on Content-based Image and Video Retrieval (CIVR '08). ACM, NY, NY, USA, 259-268. DOI: http://dx.doi.org/10.1145/1386352.1386389 Google ScholarDigital Library
- Aiden R. Doherty, Niamh Caprani, Ciaran í Conaire, Vaiva Kalnikaite, Cathal Gurrin, Alan F. Smeaton, and Noel E. O'Connor. 2011. Passively Recognising Human Activities Through Lifelogging. Comput. Hum. Behav. 27, 5 (Sept. 2011), 1948-1958. DOI: http://dx.doi.org/10.1016/j.chb.2011.05.002 Google ScholarCross Ref
- Alireza Fathi, Ali Farhadi, and James M Rehg. 2011a. Understanding egocentric activities. In Computer Vision (ICCV), 2011 IEEE International Conference on. IEEE, 407-414. DOI: http://dx.doi.org/10.1109/ICCV.2011.6126269 Google ScholarDigital Library
- Alireza Fathi, Xiaofeng Ren, and James M Rehg. 2011b. Learning to recognize objects in egocentric activities. In Computer Vision and Pattern Recognition (CVPR), 2011 IEEE Conference On. IEEE, 3281-3288. DOI: http://dx.doi.org/10.1109/CVPR.2011.5995444 Google ScholarDigital Library
- Rowanne Fleck and Geraldine Fitzpatrick. 2009. Teachers' and Tutors' Social Reflection Around SenseCam Images. Int. J. Hum.-Comput. Stud. 67, 12 (Dec. 2009), 1024-1036. DOI: http://dx.doi.org/10.1016/j.ijhcs.2009.09.004 Google ScholarDigital Library
- Jim Gemmell, Gordon Bell, and Roger Lueder. 2006. MyLifeBits: A Personal Database for Everything. Commun. ACM 49, 1 (Jan. 2006), 88-95. DOI: http://dx.doi.org/10.1145/1107458.1107460 Google ScholarDigital Library
- Jim Gemmell, Lyndsay Williams, Ken Wood, Roger Lueder, and Gordon Bell. 2004. Passive Capture and Ensuing Issues for a Personal Lifetime Store. In Proceedings of the the 1st ACM Workshop on Continuous Archival and Retrieval of Personal Experiences (CARPE'04). ACM, NY, NY, USA, 48-55. DOI: http://dx.doi.org/10.1145/1026653.1026660 Google ScholarDigital Library
- Andreas Girgensohn. 2003. A fast layout algorithm for visual video summaries. In Multimedia and Expo, 2003. ICME'03. Proceedings. 2003 International Conference on, Vol. 2. IEEE, II-77. DOI: http://dx.doi.org/10.1109/ICME.2003.1221557 Google ScholarDigital Library
- Human Performance Research Group. 1988. NASA Task Load Index (TLX) v1.0. Paper and Pencil Package. NASA Ames Research Center, Moffett Field CA (1988).Google Scholar
- Rebecca Gulotta, William Odom, Jodi Forlizzi, and Haakon Faste. 2013. Digital Artifacts As Legacy: Exploring the Lifespan and Value of Digital Data. In Proceedings of the SIGCHI Conference on Human Factors in Computing Systems (CHI '13). ACM, NY, NY, USA, 1813-1822. DOI: http://dx.doi.org/10.1145/2470654.2466240 Google ScholarDigital Library
- Cathal Gurrin, Gareth J. F. Jones, Hyowon Lee, Neil O'Hare, Alan F. Smeaton, and Noel Murphy. 2005. Mobile Access to Personal Digital Photograph Archives. In Proceedings of the 7th International Conference on Human Computer Interaction with Mobile Devices & Services (MobileHCI '05). ACM, NY, NY, USA, 311-314. DOI: http://dx.doi.org/10.1145/1085777.1085842 Google ScholarDigital Library
- R. Harper, D. Randall, N. Smyth, C. Evans, L. Heledd, and R. Moore. 2008. The Past is a Different Place: They Do Things Differently There. In Proceedings of the 7th ACM Conference on Designing Interactive Systems (DIS '08). ACM, NY, NY, USA, 271-280. DOI: http://dx.doi.org/10.1145/1394445.1394474 Google ScholarDigital Library
- Steve Hodges, Lyndsay Williams, Emma Berry, Shahram Izadi, James Srinivasan, Alex Butler, Gavin Smyth, Narinder Kapur, and Ken Wood. 2006. SenseCam: A Retrospective Memory Aid. In UbiComp 2006: Ubiquitous Computing, Paul Dourish and Adrian Friday (Eds.). Lecture Notes in Computer Science, Vol. 4206. Springer Berlin Heidelberg, 177-193. DOI: http://dx.doi.org/10.1007/11853565_11 Google ScholarDigital Library
- Benjamin Hoferlin, Markus Hoferlin, Daniel Weiskopf, and Gunther Heidemann. 2011. Information-based adaptive fast-forward for visual surveillance. Multimedia Tools and Applications 55, 1 (2011), 127-150. DOI: http://dx.doi.org/10.1007/s11042-010-0606-z Google ScholarDigital Library
- Nebojsa Jojic, Alessandro Perina, and Vittorio Murino. 2010. Structural epitome: A way to summarize ones visual experience. In Advances in neural information processing systems. 1027-1035. http://research.microsoft.com/apps/pubs/default.aspx?id=167024Google Scholar
- Jacqueline Kerr, Simon J. Marshall, Suneeta Godbole, Jacqueline Chen, Amanda Legge, Aiden R. Doherty, Paul Kelly, Melody Oliver, Hannah M. Badland, and Charlie Foster. 2013. Using the SenseCam to Improve Classifications of Sedentary Behavior in Free-Living Settings. American Journal of Preventive Medicine 44, 3 (2013), 290 - 296. DOI: http://dx.doi.org/10.1016/j.amepre.2012.11.004Google ScholarCross Ref
- Bettina Laugwitz, Theo Held, and Martin Schrepp. 2008. Construction and Evaluation of a User Experience Questionnaire. In HCI and Usability for Education and Work, Andreas Holzinger (Ed.). Lecture Notes in Computer Science, Vol. 5298. Springer Berlin Heidelberg, 63-76. DOI: http://dx.doi.org/10.1007/978--3--540--89350--9_6 Google ScholarDigital Library
- Hyowon Lee, Alan F Smeaton, Noel E OConnor, Gareth Jones, Michael Blighe, Daragh Byrne, Aiden Doherty, and Cathal Gurrin. 2008. Constructing a SenseCam visual diary as a media process. Multimedia Systems 14, 6 (2008), 341-349. DOI: http://dx.doi.org/10.1007/s00530-008-0129-x Google ScholarDigital Library
- Matthew L. Lee and Anind K. Dey. 2007. Providing Good Memory Cues for People with Episodic Memory Impairment. In Proceedings of the 9th International ACM SIGACCESS Conference on Computers and Accessibility (Assets '07). ACM, NY, NY, USA, 131-138. DOI: http://dx.doi.org/10.1145/1296843.1296867 Google ScholarDigital Library
- Matthew L. Lee and Anind K. Dey. 2008. Lifelogging Memory Appliance for People with Episodic Memory Impairment. In Proceedings of the 10th International Conference on Ubiquitous Computing (UbiComp '08). ACM, NY, NY, USA, 44-53. DOI: http://dx.doi.org/10.1145/1409635.1409643 Google ScholarDigital Library
- Cheng Li and Kris M. Kitani. 2013. Pixel-Level Hand Detection in Ego-centric Videos. In Computer Vision and Pattern Recognition (CVPR), 2013 IEEE Conference on. IEEE, 3570-3577. DOI: http://dx.doi.org/10.1109/CVPR.2013.458 Google ScholarDigital Library
- Sian E. Lindley, Richard Harper, Dave Randall, Maxine Glancy, and Nicola Smyth. 2009. Fixed in Time and "Time in Motion": Mobility of Vision Through a SenseCam Lens. In Proceedings of the 11th International Conference on Human-Computer Interaction with Mobile Devices and Services (MobileHCI '09). ACM, NY, NY, USA, Article 2, 10 pages. DOI: http://dx.doi.org/10.1145/1613858.1613861 Google ScholarDigital Library
- Catherine Loveday and Martin A Conway. 2011. Using SenseCam with an amnesic patient: Accessing inaccessible everyday memories. Memory 19, 7 (2011), 697-704. DOI: http://dx.doi.org/10.1080/09658211.2011.610803Google ScholarCross Ref
- Zheng Lu and Kristen Grauman. 2013. Story-Driven Summarization for Egocentric Video. In Computer Vision and Pattern Recognition (CVPR), 2013 IEEE Conference on. IEEE, 2714-2721. DOI: http://dx.doi.org/10.1109/CVPR.2013.350 Google ScholarDigital Library
- Martina Luchetti and Angelina R Sutin. 2015. Measuring the phenomenology of autobiographical memory: A short form of the Memory Experiences Questionnaire. Memory ahead-of-print (2015), 1-11. DOI: http://dx.doi.org/10.1080/09658211.2015.1031679Google Scholar
- Steve Mann. 1997. Wearable computing: A first step toward personal imaging. Computer 30, 2 (1997), 25-32. DOI: http://dx.doi.org/10.1109/2.566147 Google ScholarDigital Library
- Gillian O'Loughlin, Sarah Jane Cullen, Adrian McGoldrick, Siobhan O'Connor, Richard Blain, Shane O'Malley, and Giles D Warrington. 2013. Using a Wearable Camera to Increase the Accuracy of Dietary Analysis. American journal of preventive medicine 44, 3 (2013), 297-301. DOI: http://dx.doi.org/10.1016/j.amepre.2012.11.007Google Scholar
- Kadir Peker, Ajay Divakaran, and others. 2004. Adaptive fast playback-based video skimming using a compressed-domain visual complexity measure. In Multimedia and Expo, 2004. ICME'04. 2004 IEEE International Conference on, Vol. 3. IEEE, 2055-2058. DOI: http://dx.doi.org/10.1109/ICME.2004.1394669Google ScholarCross Ref
- Nemanja Petrovic, Nebojsa Jojic, and ThomasS. Huang. 2005. Adaptive Video Fast Forward. Multimedia Tools and Applications 26, 3 (2005), 327-344. DOI: http://dx.doi.org/10.1007/s11042-005-0895--9 Google ScholarDigital Library
- Hamed Pirsiavash and Deva Ramanan. 2012. Detecting activities of daily living in first-person camera views. In Computer Vision and Pattern Recognition (CVPR), 2012 IEEE Conference on. IEEE, 2847-2854. DOI: http://dx.doi.org/10.1109/CVPR.2012.6248010 Google ScholarDigital Library
- Corina Sas, Tomasz Fratczak, Matthew Rees, Hans Gellersen, Vaiva Kalnikaite, Alina Coman, and Kristina Hook. 2013. AffectCam: Arousal- Augmented Sensecam for Richer Recall of Episodic Memories. In CHI '13 Extended Abstracts on Human Factors in Computing Systems (CHI EA '13). ACM, NY, NY, USA, 1041-1046. DOI: http://dx.doi.org/10.1145/2468356.2468542 Google ScholarDigital Library
- Abigail J. Sellen, Andrew Fogg, Mike Aitken, Steve Hodges, Carsten Rother, and Ken Wood. 2007. Do Life-logging Technologies Support Memory for the Past?: An Experimental Study Using Sensecam. In Proceedings of the SIGCHI Conference on Human Factors in Computing Systems (CHI '07). ACM, NY, NY, USA, 81-90. DOI: http://dx.doi.org/10.1145/1240624.1240636 Google ScholarDigital Library
- Ana R. Silva, Salome Pinho, Luís M. Macedo, and Chris J. Moulin. 2013. Benefits of SenseCam Review on Neuropsychological Test Performance. American Journal of Preventive Medicine 44, 3 (2013), 302-307. DOI: http://dx.doi.org/10.1016/j.amepre.2012.11.005Google ScholarCross Ref
- Shingo Uchihashi, Jonathan Foote, Andreas Girgensohn, and John Boreczky. 1999. Video Manga: Generating Semantically Meaningful Video Summaries. In Proceedings of the Seventh ACM International Conference on Multimedia (Part 1) (MULTIMEDIA '99). ACM, NY, NY, USA, 383-392. DOI: http://dx.doi.org/10.1145/319463.319654 Google ScholarDigital Library
- Emma Woodberry, GAna Browne, Steve Hodges, Peter Watson, Narinder Kapur, and Ken Woodberry. 2015. The use of a wearable camera improves autobiographical memory in patients with Alzheimer's disease. Memory 23, 3 (2015), 340-349. DOI: http://dx.doi.org/10.1080/09658211.2014.886703Google ScholarCross Ref
Index Terms
- Impact of Video Summary Viewing on Episodic Memory Recall: Design Guidelines for Video Summarizations
Recommendations
Impact of reviewing lifelogging photos on recalling episodic memories
UbiComp '16: Proceedings of the 2016 ACM International Joint Conference on Pervasive and Ubiquitous Computing: AdjunctPhotos are a rich and popular form for preserving memories. Thus, they are widely used as cues to augment human memory. Near-continuous capture and sharing of photos have generated a need to summarize and review relevant photos to revive important ...
Providing good memory cues for people with episodic memory impairment
Assets '07: Proceedings of the 9th international ACM SIGACCESS conference on Computers and accessibilityAlzheimer's disease impairs episodic memory and subtly and progressively robs people of their ability to remember their recent experiences. In this paper, we describe two studies that lead to a better understanding of how caregivers use cues to support ...
Relation between facial expression and user described episodic memory timeline in short movies
ACE '16: Proceedings of the 13th International Conference on Advances in Computer Entertainment TechnologyNarrative or episodic memory is a way of how humans recall a story. Narrative has also a strong connection with human emotions and attention. We combine the analysis of facial expressions and attention to find a way to generate an episodic memory ...
Comments