MathLLMs/FigCodifier

1---2license: apache-2.03language:4- en5metrics:6- accuracy7pipeline_tag: image-text-to-text8tags:9- mathematics10- reasoning11- multi-modal-qa12- math-qa13- figure-qa14- geometry-qa15- math-word-problem16- textbook-qa17- vqa18- geometry-diagram19- synthetic-scene20- chart21- plot22- scientific-figure23- table24- function-plot25- abstract-scene26- puzzle-test27- document-image28- science29library_name: transformers30base_model:31- OpenGVLab/InternVL2-8B32datasets:33- MathLLMs/ImgCode-8.6M34---35# MathCoder-VL: Bridging Vision and Code for Enhanced Multimodal Mathematical Reasoning36 37Repo: [https://github.com/mathllm/MathCoder](https://github.com/mathllm/MathCoder)38 39Paper: [https://huggingface.co/papers/2505.10557](https://huggingface.co/papers/2505.10557)40 41 42## Introduction43We introduce MathCoder-VL, a series of open-source large multimodal models (LMMs) specifically tailored for general math problem-solving. We also introduce [FigCodifier-8B](https://huggingface.co/MathLLMs/FigCodifier), an image-to-code model trained with [ImgCode-8.6M](https://huggingface.co/datasets/MathLLMs/ImgCode-8.6M).44 45| Base Model                                          	|Ours                                               |46|-------------------------------------------------------------------|-----------------------------------------------------------------------|47|  [Mini-InternVL-Chat-2B-V1-5](https://huggingface.co/OpenGVLab/Mini-InternVL-Chat-2B-V1-5)  |  [MathCoder-VL-2B](https://huggingface.co/MathLLMs/MathCoder-VL-2B)   	|48|  [InternVL2-8B](https://huggingface.co/OpenGVLab/InternVL2-8B)  |     	[MathCoder-VL-8B](https://huggingface.co/MathLLMs/MathCoder-VL-8B)|49|  [InternVL2-8B](https://huggingface.co/OpenGVLab/InternVL2-8B)  |     	[FigCodifier-8B](https://huggingface.co/MathLLMs/FigCodifier)|50 51 52 53## Usage54For training and inference code, please refer to [InternVL2-8B](https://huggingface.co/OpenGVLab/InternVL2-8B).55 56 57 58### Prompt for TikZ Code Generation59 60```61<image>\nPlease generate the corresponding TikZ code that accurately represents the visual elements in the image. TikZ is a powerful tool for creating vector graphics within LaTeX documents. Your generated code should be precise, well-structured, and should recreate the image as faithfully as possible.62```63 64<div align="center">65  <img src="./examples/tikzimage.png" width="100%" title="Result Figure">66</div>67 68### Prompt for Python Code Generation69 70```71Please provide the Python code needed to reproduce this image.\n<image>72```73 74<div align="center">75  <img src="./examples/pyimage.png" width="100%" title="Result Figure">76</div>77 78 79## Motivation80 81<div align="center">82  <img src="./examples/fig1.png" width="100%" title="Result Figure">83</div>84 85## Construction of FigCodifier86 87<div align="center">88  <img src="./examples/fig2.png" width="100%" title="Result Figure">89</div>90 91 92 93## **Citation**94 95Please cite the paper if you use our data, model or code.96 97```98@inproceedings{99wang2025mathcodervl,100title={MathCoder-{VL}: Bridging Vision and Code for Enhanced Multimodal Mathematical Reasoning},101author={Ke Wang and Junting Pan and Linda Wei and Aojun Zhou and Weikang Shi and Zimu Lu and Han Xiao and Yunqiao Yang and Houxing Ren and Mingjie Zhan and Hongsheng Li},102booktitle={The 63rd Annual Meeting of the Association for Computational Linguistics},103year={2025},104url={https://openreview.net/forum?id=nuvtX1imAb}105}106 107@inproceedings{108wang2024mathcoder,109title={MathCoder: Seamless Code Integration in {LLM}s for Enhanced Mathematical Reasoning},110author={Ke Wang and Houxing Ren and Aojun Zhou and Zimu Lu and Sichun Luo and Weikang Shi and Renrui Zhang and Linqi Song and Mingjie Zhan and Hongsheng Li},111booktitle={The Twelfth International Conference on Learning Representations},112year={2024},113url={https://openreview.net/forum?id=z8TW0ttBPp}114}115```