Research / publications & ideas

    Research & Publications.

    Filter by Topic
    Filter by Year

    Publications

    Featured Research
    AI Agents

    Emergence World: Adversarial Stress-Testing of Long-Horizon Multi-Agent Systems

    D Akkil, T Abuelsaad, K Vikram, M Pace, A Vempaty, S Beotra, R Kokku, et al.

    Published in: arXiv preprint arXiv:2609.17320

    1. AI Agents

      Emergence World: A Platform for Evaluating Long-Horizon Multi-Agent Autonomy

      D Akkil, R Kokku, K Vikram, T Abuelsaad, A Vempaty, S Nitta

      arXiv preprint arXiv:2606.08367

    2. AI Agents

      Emergence WebVoyager: Toward Consistent and Transparent Evaluation of (Web) Agents in The Wild

      D Akkil, M Allaham, A Raj, T Abuelsaad, R Kokku

      arXiv preprint arXiv:2603.29020

    3. HCI

      Navigating the Future: A Design Fiction Study on User Perceptions of Next-Gen LLM-Based Voice Interaction

      B Thankachan, D Akkil, S Rahman, K Jokinen, M Turunen

      Multimodal Technologies and Interaction 10(3), 31

    4. AI Agents

      Agent-E: From Autonomous Web Navigation to Foundational Design Principles in Agentic Systems

      T Abuelsaad, D Akkil, P Dey, A Jagmohan, A Vempaty, R Kokku

      arXiv preprint arXiv:2407.13032

    5. AI Agents

      MathViz-E: A case-study in domain-specialized tool-using agents

      A Bulusu, B Man, A Jagmohan, A Vempaty, J Mari-Wyka, D Akkil

      arXiv preprint arXiv:2407.17544

    6. Eye-tracking

      Toward a real-time index of pupillary activity as an indicator of cognitive load

      G Jayawardena, Y Jayawardana, S Jayarathna, J Högström, T Papa, D Akkil, et al.

      Procedia Computer Science 207, 1331-1340

    7. HCI

      Gaze interaction with vibrotactile feedback: Review and design guidelines

      J Rantala, P Majaranta, J Kangas, P Isokoski, D Akkil, O Špakov, et al.

      Human–Computer Interaction 35 (1), 1-39

    8. Kinesthetic interaction

      Gaze-based kinaesthetic interaction for virtual reality

      Z Li, D Akkil, R Raisamo

      Interacting with Computers 32 (1), 17-32

    9. Kinesthetic interaction

      The impact of control-display gain in kinesthetic search

      Z Li, D Akkil, R Raisamo

      International Conference on Human Haptic Sensing and Touch Enabled Computer Applications

    10. Kinesthetic interaction

      Gaze augmented hand-based kinesthetic interaction: What you see is what you feel

      Z Li, D Akkil, R Raisamo

      IEEE Transactions on Haptics 12 (2), 114-127

    11. Eye-tracking

      I see what you see: gaze awareness in mobile video collaboration

      D Akkil, B Thankachan, P Isokoski

      ACM Symposium on Eye Tracking Research & Applications

    12. Eye-tracking

      Comparison of gaze and mouse pointers for video-based collaborative physical task

      D Akkil, P Isokoski

      Interacting with Computers 30 (6), 524-542

    13. HCI

      Where the streets have no name: a field trip in the wild

      B Thankachan, S Sharma, T Gross, D Akkil, M Turunen, S Mehrotra, et al.

      Interactions 25 (3), 56-60

    14. HCI

      Vibrotactile stimulation of the head enables faster gaze gestures

      J Kangas, J Rantala, D Akkil, P Isokoski, P Majaranta, R Raisamo

      International Journal of Human-Computer Studies 98, 62-71

    15. AI Agents

      Gaze awareness in agent-based early-childhood learning application

      D Akkil, P Dey, D Salian, N Rajput

      IFIP Conference on Human-Computer Interaction

    16. AI Agents

      Little bear–a gaze aware learning companion for early childhood learners

      D Akkil, P Dey, N Rajput

      IFIP Conference on Human-Computer Interaction

    17. Eye-tracking

      GazeTorch: enabling gaze awareness in collaborative physical tasks

      D Akkil, JM James, P Isokoski, J Kangas

      CHI Conference Extended Abstracts

    18. Eye-tracking

      Feedback for smooth pursuit gaze tracking based control

      J Kangas, O Špakov, P Isokoski, D Akkil, J Rantala, R Raisamo

      Augmented Human International Conference

    19. HCI

      Evaluation of HeadTurn: An interaction technique using the gaze and head turns

      T Nukarinen, J Kangas, O Špakov, P Isokoski, D Akkil, J Rantala, et al.

      Nordic Conference on Human-Computer Interaction

    20. Eye-tracking

      PursuitAdjuster: an exploration into the design space of smooth pursuit-based widgets

      O Špakov, P Isokoski, J Kangas, D Akkil, P Majaranta

      ACM Symposium on Eye Tracking Research & Applications

    21. Eye-tracking

      Gaze augmentation in egocentric video improves awareness of intention

      D Akkil, P Isokoski

      CHI Conference on Human Factors in Computing Systems

    22. HCI

      Accuracy of interpreting pointing gestures in egocentric view

      D Akkil, P Isokoski

      ACM International Joint Conference on Pervasive and Ubiquitous Computing

    23. Wearables

      User expectations of everyday gaze interaction on smartglasses

      D Akkil, A Lucero, J Kangas, T Jokela, M Salmimaa, R Raisamo

      Nordic Conference on Human-Computer Interaction

    24. Eye-tracking

      Haptic feedback in eye typing

      P Majaranta, P Isokoski, J Rantala, O Špakov, D Akkil, J Kangas, et al.

      Journal of Eye Movement Research 9 (1)

    25. HCI

      Comparison of three implementations of HeadTurn: a multimodal interaction technique with gaze and head turns

      O Špakov, P Isokoski, J Kangas, J Rantala, D Akkil, R Raisamo

      ACM International Conference on Multimodal Interaction

    26. HCI

      Both fingers and head are acceptable in sensing tactile feedback of gaze gestures

      J Kangas, J Rantala, D Akkil, P Isokoski, P Majaranta, R Raisamo

      International Conference on Human Haptic Sensing and Touch Enabled Computer Applications

    27. Wearables

      Glance awareness and gaze interaction in smartwatches

      D Akkil, J Kangas, J Rantala, P Isokoski, O Spakov, R Raisamo

      CHI Conference Extended Abstracts

    28. Wearables

      Haptic feedback of gaze gestures with glasses: localization accuracy and effectiveness

      J Rantala, J Kangas, P Isokoski, D Akkil, O Špakov, R Raisamo

      ACM International Joint Conference on Pervasive and Ubiquitous Computing

    29. Eye-tracking

      Gaze gestures and haptic feedback in mobile devices

      J Kangas, D Akkil, J Rantala, P Isokoski, P Majaranta, R Raisamo

      CHI Conference on Human Factors in Computing Systems

    30. Wearables

      Glasses with haptic feedback of gaze gestures

      J Rantala, J Kangas, D Akkil, P Isokoski, R Raisamo

      CHI'14 Extended Abstracts

    31. Eye-tracking

      TraQuMe: a tool for measuring the gaze tracking quality

      D Akkil, P Isokoski, J Kangas, J Rantala, R Raisamo

      Symposium on Eye Tracking Research and Applications

    32. Wearables

      Using gaze gestures with haptic feedback on glasses

      J Kangas, D Akkil, J Rantala, P Isokoski, P Majaranta, R Raisamo

      Nordic Conference on Human-Computer Interaction

    33. HCI

      Delayed haptic feedback to gaze gestures

      J Kangas, J Rantala, D Akkil, P Isokoski, P Majaranta, R Raisamo

      International Conference on Human Haptic Sensing and Touch Enabled Computer Applications

    View All Publications on Google Scholar

    This list is also available as Markdown, JSON and BibTeX.

    Patents.

    4 granted patents and 2 pending applications in eye-tracking, gaze interaction, AI agents, and voice-based digital assistant technologies.

    Granted Patents (4)

    • Granted
      1 cited

      Context-sensitive customization of remote-control unit

      US12,229,354

      D Akkil, P Dey, R Kokku, SS Ponoth, HI Alonso, S O'Hara, SV Nitta

      IBM
      View Patent
    • Granted

      Content transfer among remote-control units and integrated output devices

      US12,425,460

      D Akkil, P Dey, R Kokku, SS Ponoth, HI Alonso, S O'Hara, SV Nitta

      IBM
      View Patent
    • Granted
      33 cited

      Digital assistant and method for voice-based interactive communication based on detected user gaze indicating attention

      US11,699,439

      E George-Svahn, S Pateriya, O Kurt, D Akkil

      Tobii AB
      View Patent
    • Granted
      2 cited

      System and method for determining reference gaze data

      US12,527,474

      H Andersson, D Akkil

      Tobii AB
      View Patent

    Pending Applications (2)

    • Pending

      Intelligent web-based task planning and execution

      US App. 18/625,088

      A Jagmohan, P Dey, SV Nitta, R Kokku, A Vempaty, T Abuelsaad, D Akkil

      IBM
      View Application
    • Pending

      Decoupled secure data handling for generative AI applications

      US App. 19/196,391

      T Abuelsaad, D Akkil, R Kokku, P Dey

      Merlyn Mind
      View Application

    Blog & Insights.

    Thoughts on AI, research findings, and insights from building autonomous agents and human-computer interaction systems.

    Featured Post
    8 min read

    Agent-E SOTA Results on WebVoyager Benchmark

    We're excited to share that Agent-E has achieved state-of-the-art results on the WebVoyager benchmark, outperforming existing text and multi-modal web agents by 10-30%.

    AI Agents
    Research
    Web Automation
    Read Article
    • 10 min read

      Trust by Design: Our UX Principles for Building Trustable Data Agents

      Five UX principles for trustworthy data agents: thinking aloud, collaborating with users, surfacing assumptions, explaining analytical logic, and reusing human-verified workflows.

      AI Agents
      UX
      Trust
      Read More
    • 6 min read

      Distilling the Web for Multi-Agent Automation

      Exploring advanced DOM distillation and denoising methods that enable LLMs to better understand and interact with web pages.

      AI
      Web
      LLM
      Read More
    • 7 min read

      100 Years of Eye-Gaze Tracking

      The technology of eye gaze tracking has gone through over a century of research and development. Exploring its journey from invasive devices requiring anaesthesia to mainstream AR/VR tech.

      Eye-tracking
      VR
      AR
      Technology
      Read More