Broadly speaking, we work at the intersection of computational neuroscience and machine learning, aka AI4(Neuro)Science. Ultimately, we are interested in reverse engineering the algorithms of the brain, in order to figure out how the brain works and to build better artificial intelligence systems.
Check out group's website for more information, and see our open source code below!
We also share open data/model weights on Zenodo and Huggingface!
- DeepLabCut: for animal pose estimation
- DLC2action: for action segmentation
- hBehaveMAE: unsupervised action decomposition for hierarchical behavior
- LLaVAction: multimodal language model for action recognition
Skill learning (MyoChallenges @NeurIPS):
- Winning code for the penalty kick track of the MyoChallenge at NeurIPS 2025, Code by Simos, Li & Ziliotto.
- Winning code for the object manipulation track of the MyoChallenge at NeurIPS 2023, Code by Marin Vargas & Chiappa.
- Winning code for the Baoding ball MyoChallenge at NeurIPS 2022, joint work with Pouget Lab (University of Geneva). Caggiano et al. Proceedings of Machine Learning Research 2022
Computer Vision and Behavioral Analysis:
- LLaVAction: evaluating and training multi-modal large language models for action recognition: Qi*, Ye*, Mathis**, Mathis**, ICLR 2026 (accepted)
- Elucidating the Hierarchical Nature of Behavior with Masked Autoencoders: Stoffl, Bonnetto, d'Ascoli & Mathis ECCV 2024
- HOISDF: Constraining 3D Hand-Object Pose Estimation with Global Signed Distance Fields: Code for Haozhe Qi, Chen Zhao, Mathieu Salzmann, & Alexander Mathis. CVPR 2024
- WildCLIP: Scene and animal attribute retrieval from camera trap data with domain-adapted vision-language models: Code for Gabeff, Russwurm, Tuia & Mathis International Journal of Computer Vision 2024 (also oral at CVPR CV4animals 2023)
- Bottom-up conditioned top-down pose estimation (BUCTD): Code for Zhou*, Stoffl*, Mathis and Mathis ICCV 2023. State of the art code for performing 2D pose estimation in crowded scenes.
- End-to-end trainable multi-instance pose estimation with transformers: Code for POET model, Stoffl, Vidal & Mathis arxiv 2021
- AcinoSet: A 3D Pose Estimation Dataset and Baseline Models for Cheetahs in the Wild, Joska et al. ICRA 2021
- Primer on Motion Capture, Mathis et al. Neuron 2020
AI4Science including modeling proprioception and sensorimotor control:
- Deep-learning models of the ascending proprioceptive pathway are subject to illusions: Code for modeling proprioceptive illusions. Adriana Perez Rotondo, Merkourios Simos, Florian David, Sebastian Pigeon, Olaf Blanke, & Alexander Mathis. Experimental Physiology 2025
- Task-driven-proprioception: Code for modeling the proprioceptive system of primates. Marin Vargas* & Bisi* et al. Cell 2024
- ODEformer: symbolic regression of dynamical systems with transformers: Code from d'Ascoli*, Becker*, Mathis, Schwaller & Kilbertus ICLR 2024 (spotlight). Cool code to infer symbolic formulas from data
- DeepDraw: Code for modeling proprioception with task-driven modeling, Sandbrink*, Mamidanna* et al. eLife 2023
Reinforcement learning (mostly for motor skills also relevant for modeling sensorimotor control):
- Arnold: a generalist muscle transformer policy: Code for Chiappa & An et al., arxiv 2025
- Reinforcement Learning-Based Motion Imitation for Physiologically Plausible Musculoskeletal Motor Control: Code for Simos, Chiappa & Mathis, ICRA 2026 (accepted)
- Acquiring musculoskeletal skills with curriculum-based reinforcement learning: Code for Chiappa*, Tano*, Patel* et al. Neuron 2024
- Latent Exploration for Reinforcement Learning: Code for Chiappa et al. NeurIPS 2023
- DMAP: a Distributed Morphological Attention Policy for Learning to Locomote with a Changing Body: Code for Chiappa, Marin-Vargas & Mathis NeurIPS 2022
Also check out the section on winning MyoChallenges at NeurIPS (in 2022, 2023 and 2025)!
Datasets and benchmarks:
- EPFL-Smart-Kitchen-30: Densely annotated cooking dataset with 3D kinematics to challenge video and language models: Code and data for Bonnetto*, Qi* et al. NeurIPS 2025
- Behavior understanding in ecology benchmark: Code for MammAlps. Gabeff et al. CVPR spotlight 2025
🌈 Please reach out, if you want to work with us! We love collaborative, open-source science.
We often collaborate with the group of Mackenzie Mathis, and also recommend checking out their GitHub repository!
