Börje F. Karlsson (申博野)

Principal Member of Technical Staff
Beijing Academy of Artificial Intelligence (BAAI)

email:	borje{dot}karlsson{at}tellarin.com
website:	Börje F. Karlsson
address:	Beijing Academy of Artificial Intelligence 150 Chengfu Road, 5F Haidian District Beijing, P.R. China 100080
links:

Short Biography

I am currently Principal Member of Technical Staff at the Beijing Academy of Artificial Intelligence (BAAI), a non-profit research insitute in Beijing, China, where I work on new AI scenarios involving intelligent agents and multi-modality models. My main interests include machine learning systems, multi-modality reasoning, intelligent agents, multilingual knowledge pipelines, and digital entertainment.

From 2012 to early 2023, I was Principal Research Engineer Lead at Microsoft Research Asia (MSRA). Initially a member of the Mobile and Sensing Systems and System Algorithms research groups, I later joined the Knowledge Computing group – part of the Data, Knowledge, and Intelligence (DKI) research area.

Previously I was a Research Scientist at Nokia Institute of Technology (INdT) - a Nokia-founded R&D lab in Brazil, where I led its Contextual Platform group. Previously I’ve also worked as system/software architect, consultant, product owner, and development team lead on projects for companies such as Nokia, C.E.S.A.R., Motorola, 3, TIM, Radix.com, Petrobras, among others.

I received my doctorate in Computer Science from PUC-Rio, Brazil in 2010 and my BSc from UFPE, also in Brazil.

I've also been a member of the organizing committee and TPC of conferences, including AAAI, ACL, IJCAI, MobileSoft, SIGGRAPH Asia's MGIA, IFIP ICEC, etc.

» students Research

My research interests lie broadly in machine learning systems, knowledge mining, mobile systems, pervasive computing, and digital entertainment, with a recent focus on agentic systems, embodied egocentric AI, and multi-cultural multimodal models.

Previous projects include: tabular data semantics, design intelligence, modular KBQA, real-world information extraction, and location-based systems (e.g., navigation, mapping, etc.).

» full list Selected publications

[ICLR'25] MLLM as Retriever: Interactively Learning Multimodal Retrieval for Embodied Agents [paper] [bib]

Junpeng Yue, Xinrun Xu, Börje F. Karlsson, Zongqing Lu.
International Conference on Learning Representations, 2025

[ICLR'25] INCLUDE: Evaluating Multilingual Language Understanding with Regional Knowledge [paper] [bib]

Angelika Romanou, Negar Foroutan, Anna Sotnikova, Sree Harsha Nelaturu, Shivalika Singh, Rishabh Maheshwary, Micol Altomare, Zeming Chen, Mohamed A. Haggag, Snegha A, Alfonso Amayuelas, Azril Hafizi Amirudin, Danylo Boiko, Michael Chang, Jenny Chim, Gal Cohen, Aditya Kumar Dalmia, Abraham Diress, Sharad Duwal, Daniil Dzenhaliou, Daniel Fernando Erazo Florez, Fabian Farestam, Joseph Marvin Imperial, Shayekh Bin Islam, Perttu Isotalo, Maral Jabbarishiviari, Börje F. Karlsson, Eldar Khalilov, Christopher Klamm, Fajri Koto, Dominik Krzemiński, Gabriel Adriano de Melo, Syrielle Montariol, Yiyang Nan, Joel Niklaus, Jekaterina Novikova, Johan Samir Obando Ceron, Debjit Paul, Esther Ploeger, Jebish Purbey, Swati Rajwal, Selvan Sunitha Ravi, Sara Rydell, Roshan Santhosh, Drishti Sharma, Marjana Prifti Skenduli, Arshia Soltani Moakhar, Bardia Soltani Moakhar, Ayush Kumar Tarun, Azmine Toushik Wasi, Thenuka Ovin Weerasinghe, Serhan Yilmaz, Mike Zhang, Imanol Schlag, Marzieh Fadaee, Sara Hooker, Antoine Bosselut.
International Conference on Learning Representations, 2025

[arXiv, OWA @ NeurIPS'24] Cradle: Empowering Foundation Agents Towards General Computer Control [paper] [bib]

Weihao Tan, Wentao Zhang, Xinrun Xu, Haochong Xia, Ziluo Ding, Boyu Li, Bohan Zhou, Junpeng Yue, Jiechuan Jiang, Yewen Li, Ruyi An, Molei Qin, Chuqiao Zong, Longtao Zheng, Yujie Wu, Xiaoqiang Chai, Yifei Bi, Tianbao Xie, Pengjie Gu, Xiyun Li, Ceyao Zhang, Long Tian, Chaojie Wang, Xinrun Wang, Börje F. Karlsson, Bo An, Shuicheng Yan, Zongqing Lu.
arXiv:arXiv:2403.03186, July 2024
NeurIPS Workshop on Open-World Agents: Synnergizing Reasoning and Decision-Making in Open-World Environments (OWA-2024)

[EMNLP'24] SEACrowd: A Multilingual Multimodal Data Hub and Benchmark Suite for Southeast Asian Languages [paper] [bib]

Holy Lovenia, Rahmad Mahendra, Salsabil Maulana Akbar, Lester James V. Miranda, Jennifer Santoso, Elyanah Aco, Akhdan Fadhilah, Jonibek Mansurov, Joseph Marvin Imperial, Onno P. Kampman, Joel Ruben Antony Moniz, Muhammad Ravi Shulthan Habibi, Frederikus Hudi, Railey Montalan, Ryan Ignatius, Joanito Agili Lopo, William Nixon, Börje F. Karlsson, James Jaya, Ryandito Diandaru, Yuze Gao, Patrick Amadeus, Bin Wang, Jan Christian Blaise Cruz, Chenxi Whitehouse, Ivan Halim Parmonangan, Maria Khelli, Wenyu Zhang, Lucky Susanto, Reynard Adha Ryanda, Sonny Lazuardi Hermawan, Dan John Velasco, Muhammad Dehan Al Kautsar, Willy Fitra Hendria, Yasmin Moslem, Noah Flynn, Muhammad Farid Adilazuarda, Haochen Li, Johanes Lee, R. Damanhuri, Shuo Sun, Muhammad Reza Qorib, Amirbek Djanibekov, Wei Qi Leong, Quyet V. Do, Niklas Muennighoff, Tanrada Pansuwan, Ilham Firdausi Putra, Yan Xu, Ngee Chia Tai, Ayu Purwarianti, Sebastian Ruder, William Tjhi, Peerat Limkonchotiwat, Alham Fikri Aji, Sedrick Keh, Genta Indra Winata, Ruochen Zhang, Fajri Koto, Zheng-Xin Yong, Samuel Cahyawijaya.
Conference on Empirical Methods in Natural Language Processing, 2024

[LLM Agents @ ICLR'24, BAAI Tech Report] Towards General Computer Control: A Multimodal Agent for Red Dead Redemption II as a Case Study [paper] [bib]

Weihao Tan, Ziluo Ding, Wentao Zhang, Boyu Li, Bohan Zhou, Junpeng Yue, Haochong Xia, Jiechuan Jiang, Longtao Zheng, Xinrun Xu, Yifei Bi, Pengjie Gu, Xinrun Wang, Börje F. Karlsson, Bo An, Zongqing Lu.
ICLR 2024 Workshop on LLM Agents, May 2024
Beijing Academy of Artificial Intelligence Technical Report, February 2024

[TASLP] Decomposed Meta-Learning for Few-Shot Sequence Labeling [paper] [bib]

Tingting Ma, Qianhui Wu, Huiqiang Jiang, Jieru Lin, Börje F. Karlsson, Tiejun Zhao, Chin-Yew Lin.
IEEE/ACM Transactions on Audio, Speech, and Language Processing (TASLP), February 2024

[ACL'24] Aya Dataset: An Open-Access Collection for Multilingual Instruction Tuning [paper] [bib]

Shivalika Singh*, Freddie Vargus*, Daniel D’souza*, Börje F. Karlsson*, Abinaya Mahendiran*, Wei-Yin Ko*, Herumb Shandilya*, Jay Patel, Deividas Mataciunas, Laura OMahony, Mike Zhang, Ramith Hettiarachchi, Joseph Wilson, Marina Machado, Luisa Souza Moura, Dominik Krzemiński, Hakimeh Fadaei, Irem Ergün, Ifeoma Okoh, Aisha Alaagib, Oshan Mudannayake, Zaid Alyafeai, Vu Minh Chien, Sebastian Ruder, Surya Guthikonda, Emad A Alghamdi, Sebastian Gehrmann, Niklas Muennighoff, Max Bartolo, Julia Kreutzer, Ahmet Üstün, Marzieh Fadaee, Sara Hooker.
Annual Meeting of the Association for Computational Linguistics , 2024

[arXiv] A Survey on Game Playing Agents and Large Models: Methods, Applications, and Challenges [paper] [bib]

Xinrun Xu, Yuxin Wang, Chaoyi Xu, Ziluo Ding, Jiechuan Jiang, Zhiming Ding, Börje F. Karlsson.
arXiv:2403.10249, February 2024

[NAACL'24] Universal NER: A Gold-Standard Multilingual Named Entity Recognition Benchmark [paper] [bib]

Stephen Mayhew, Terra Blevins, Shuheng Liu, Marek Šuppa, Hila Gonen, Joseph M. Imperial, Börje F. Karlsson, Peiqin Lin, Nikola Ljubešic, LJ Miranda, Barbara Plank, Arij Riabi, Yuval Pinter.
Conference of the North American Chapter of the Association for Computational Linguistics, 2024

[IJCAI'24] AutoAgents: A Framework for Automatic Agent Generation [paper] [bib]

Guangyao Chen, Siwei Dong, Yu Shu, Ge Zhang, Jaward Sesay, Börje F. Karlsson, Jie Fu, Yemin Shi.
International Joint Conference on Artificial Intelligence, 2024

[ACL'23] CoLaDa: A Collaborative Label Denoising Framework for Cross-lingual Named Entity Recognition [paper] [bib]

Tingting Ma, Qianhui Wu, Huiqiang Jiang, Börje F. Karlsson, Tiejun Zhao, Chin-Yew Lin.
Annual Meeting of the Association for Computational Linguistics, 2023

[MSR Tech Report] Dataset and Baseline System for Multi-lingual Extraction and Normalization of Temporal and Numerical Expressions [paper] [bib]

Sanxing Chen, Yongqiang Chen, Börje F. Karlsson.
Microsof Research Technical Report MSR-TR-2023-9, February 2023

[EMNLP'22] TIARA: Multi-grained Retrieval for Robust Question Answering over Large Knowledge Bases [paper] [bib]

Yiheng Shu, Zhiwei Yu, Yuhan Li, Börje F. Karlsson, Tingting Ma, Yuzhong Qu, Chin-Yew Lin.
Conference on Empirical Methods in Natural Language Processing, 2022

[ACL'21] AdvPicker: Effectively Leveraging Unlabeled Data via Adversarial Discriminator for Cross-Lingual NER [paper] [bib]

Weile Chen, Huiqiang Jiang, Qianhui Wu, Börje F. Karlsson, Yi Guan.
Annual Meeting of the Association for Computational Linguistics, 2021

[IJCAI'20] UniTrans: Unifying Model Transfer and Data Transfer for Cross-Lingual Named Entity Recognition with Unlabeled Data [paper] [bib]

Qianhui Wu, Zijia Lin, Börje F. Karlsson, Biqing Huang, Jian-Guang Lou.
International Joint Conferences on Artifical Intelligence, 2020

[ACL'20] Single-/Multi-Source Cross-Lingual NER via Teacher-Student Learning on Unlabeled Data in Target Language [paper] [bib]

Qianhui Wu, Zijia Lin, Börje F. Karlsson, Jian-Guang Lou, Biqing Huang.
Annual Meeting of the Association for Computational Linguistics, 2020

[AAAI'19] GRN: Gated Relation Network to Enhance Convolutional Neural Network for Named Entity Recognition [paper] [bib]

Hui Chen, Zijia Lin, Guiguang Ding, Jian-Guang Lou, Yusen Zhang, Börje F. Karlsson.
AAAI Conference on Artificial Intelligence, 2019

[NAACL'19] CAN-NER: Convolutional Attention Network for Chinese Named Entity Recognition [paper] [bib]

Yuying Zhu, Guoxin Wang, Börje F. Karlsson.
Conference of the North American Chapter of the Association for Computational Linguistics, 2019

[INFOCOM'19] Incrementally-deployable Indoor Navigation with Automatic Trace Generation [paper] [bib]

Yuanchao Shu, Zhuqi Li, Börje F. Karlsson, Yiyong Lin, Thomas Moscibroda, Kang Shin.
IEEE International Conference on Computer Communications, 2019

[MobiCom'17] Demo: Towards Flexible and Scalable Indoor Navigation [paper] [bib]

Zhuqi Li, Yuanchao Shu, Börje F Karlsson, Yiyong Lin, Thomas Moscibroda.
ACM International Conference on Mobile Computing and Networking, 2017

[Microsoft Research Blog] Path Guide: A New Approach to Indoor Navigation [webpage]

Yuanchao Shu, and Börje F. Karlsson.
Microsoft Research Blog, July 14, 2017

[BuildSys'16] Systematically Debugging IoT Control System Correctness for Building Automation [paper] [bib]

Chieh-Jan Mike Liang, Lei Bu, Zhao Li, Junbei Zhang, Shi Han, Börje F. Karlsson, Dongmei Zhang, Feng Zhao.
ACM International Conference on Systems for Energy-Efficient Built Environments, 2016

[IPSN'15] SIFT: Building an Internet of Safe Things [paper] [bib]

Chieh-Jan Mike Liang, Börje F. Karlsson, Nicholas D Lane, Feng Zhao, Junbei Zhang, Zheyi Pan, Zhao Li, Yong Yu.
ACM International Conference on Information Processing in Sensor Networks, 2015

[MobiCom'14] Caiipa: Automated Large-scale Mobile App Testing through Contextual Fuzzing [paper] [bib]

Chieh-Jan Mike Liang, Nicholas D Lane, Niels Brouwers, Li Zhang, Börje F. Karlsson, Hao Liu, Yan Liu, Jun Tang, Xiang Shan, Ranveer Chandra, Feng Zhao.
ACM International Conference on Mobile Computing and Networking, 2014

Academic Service (recent)

Organizing Committees

ACM SIGGRAPH Asia’s Symposium on Mobile Graphics and Interactive Applications (MGIA) – Co-chair 2016, Chair 2014, Chair 2013

SBC SBGames 2013 – Tutorials Co-chair

ACM SIGGRAPH 2012 – Operations Sub-committee

Technical Program Committees

2024: IJCAI PC, ACL RR, NAACL, ACL, IFIP ICEC

2023: AAAI, IJCAI PC Board, ACL, EACL, AACL, ACL RR

2022: AAAI, IJCAI PC Board, ACL, NAACL, EMNLP, AACL, ACL RR

2021: AAAI, IJCAI Senior PC, ACL

2020: ACL, AACL-IJCNLP

2018: IEEE/ACM MobileSoft, IFIP ICEC, SIGGRAPH Asia – Emerging Technologies

2017: IEEE/ACM MobileSoft, ACM SIGGRAPH Asia’s MGIA, ACM Multimedia – Systems track

2016: IEEE CPSCom

2015: IEEE/ACM MobileSoft, IFIP ICEC, ACM SIGGRAPH Asia’s MGIA

2014: IEEE/ACM MobileSoft, IFIP ICEC

2013: IFIP ICEC

Guest co-editor — Computer & Graphics, Special Section on Recent Mobile Graphics Research, 2016/2017

Reviewer

Journals:
ACM/IEEE TASLP (2023-), Journal of Open Source Software (2023-), Neurocomputing (2021-), Entertainment Computing (2023, 2016, 2015), IEEE Transactions on Mobile Computing (2014-2018), ACM IMWUT – Interactive, Mobile, Wearable and Ubiquitous Technologies (2017), SBC RITA (2015, 2006-2011)

Conferences and Workshops:
ICLR (2024), Tiny Papers @ ICLR (2024), AfricaNLP Workshop @ ICLR (2024), Generative Models for Decision Making Workshop - GenAI4DM @ ICLR (2024)
AAAI (2019, 2017, 2014, 2010), IJCAI (2018, 2013, 2009), IEEE WoWMoM (2015), Mobiquitous (2014), MobiCASE (2013)

In my free time, I enjoy travelling and trying different foods and beer.

Site design originated from Alex Halderman.
Last modified: Mar. 2023