Multimodal Evaluation & Metrics
Building reliable evaluation protocols for vision-language models, with a focus on grounding and cultural nuance in Japanese contexts.
Research Themes
Instruction-Tuned Multimodal LLMs
Curating instruction datasets and training recipes that keep open-weight models aligned and deployable.
Structured Generation & Understanding
Teaching models to produce and consume structured outputs such as captions, diagrams, and graphs for real-world tasks.
Featured Publications
COM Kitchens: An Unedited Overhead-view Procedural Videos Dataset as a Vision-Language Benchmark
Proceedings of The 18th European Conference on Computer Vision (ECCV 2024) · 2024
Introducing a new vision-language dataset based on unedited overhead-view procedural cooking videos.
Query-based Image Captioning from Multi-context 360-degree Images
Findings of the Association for Computational Linguistics: EMNLP 2023 · 2023
A novel image captioning approach that leverages queries and multi-context 360-degree imagery.
IMPARA: Impact-Based Metric for GEC Using Parallel Data
Proceedings of the 29th International Conference on Computational Linguistics (COLING 2022) · 2022
Proposal of a new impact-based metric for grammatical error correction using parallel datasets.
Recent Publications
Why We Build Local Large Language Models: An Observational Analysis from 35 Japanese and Multilingual LLMs
The 1st Workshop on Multilingual and Equitable Language Technologies (MELT) · 2025
Building instruction-tuning datasets from human-written instructions with open-weight large language models
The 2nd Conference on Language Modeling (COLM) · 2025
Constructing Multimodal Datasets from Scratch for Rapid Development of a Japanese Visual Language Model
Proceedings of the 2025 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies (Volume 3: System Demonstrations) · 2025
Methodology for quickly constructing multimodal datasets tailored for Japanese vision-language models.
Talks & Workshops
Education
2024 – Present
Ph.D., Tokyo Institute of Technology
Vision and Language, Evaluation
Advisors: Naoaki Okazaki
Focusing on novel evaluation metrics for multimodal systems and cross-modal representation learning.
2022 – 2024
M.Eng., Tokyo Institute of Technology
Vision and Language: Image Captioning
Advisors: Naoaki Okazaki
Developed context-aware image captioning models that generate descriptions based on user preferences.
2018 – 2022
B.Eng., Tokyo Institute of Technology
NLP: Grammatical Error Correction
Advisors: Naoaki Okazaki, Masahiro Kaneko (Mentor)
Created improved evaluation metrics for grammatical error correction systems.
Awards & Fellowships
- Young Scientist Award, ANLP (2025)
- Committee Special Awarded Paper, ANLP (2025, 2023)
- Program for Development of Co-creative Experts towards Top-level AI Research (Science Tokyo BOOST) for Science and Engineering fields (2024–2027)
- Awarded Paper, ANLP (2022)
Skills & Expertise
Programming
- Python
- PyTorch
- Java
Research Areas
- Computer Vision
- Natural Language Processing
- Multimodal Learning
- Image Captioning
- Evaluation Metrics
Languages
- Japanese (Native)
- English (Professional)