| Rank | Discriminative Prediction | Generative Reasoning |
|---|---|---|
| {rank} | {prediction} | {reasoning} |
{
event.currentTarget.style.display = "none";
}}
/>
);
}
function Nav({ page, setPage }) {
const tabs = ["Home", "About", "Quickstart", "Leaderboard"];
return (
{paper.abstract}
{paper.contact}
{paper.bibtex}VT-Bench is a unified benchmark for visual-tabular multi-modal learning. It covers both discriminative prediction and generative reasoning, aiming to evaluate whether models can reliably combine image evidence with structured tabular data.
| Item | Value | Description |
|---|---|---|
| {row[0]} | {row[1]} | {row[2]} |
Predicts a discrete label or continuous value from an aligned image and a single-row tabular feature vector.
Answers natural-language questions using images and structured tables or databases, often requiring evidence localization and multi-step reasoning.
The source links and availability follow the project README. For constructed datasets, the table links to the corresponding construction scripts in the GitHub repository.
| Dataset | Identifier | Availability | Source | Task Family |
|---|---|---|---|---|
| {row[0]} | {row[1]} | {row[2]} | {row[3]} | {row[5]} |
VT-Bench supports vision unimodal models, tabular unimodal models, vision-tabular multi-modal fusion models, and general-purpose VLMs.
{Object.entries(models).map(([group, items]) => (VT-Bench provides a unified evaluation interface for both discriminative prediction and generative reasoning tasks. The core arguments are task, dataset, model, setting, and diagnostics.
{quickstartInstall}The default environment is suitable for most benchmark experiments. Due to version incompatibilities, please use the VLM-specific environment if you want to fine-tune Vision-Language Models.
{quickstartVlmInstall}Specify five arguments: task, dataset, model, setting, and diagnostics. For VLMs, use the Hugging Face full identifier, such as Qwen/Qwen3-VL-8B-Instruct.
| Argument | Description | Options / Notes |
|---|---|---|
--task | Evaluation task type. | prediction or reasoning |
--dataset | Dataset identifier. | Choose from the Datasets section, e.g., skin, ehrxqa. |
--model | Model name. | Built-in model name or Hugging Face full identifier for VLMs. |
--setting | Evaluation setting. | Varies across datasets; EHRXQA supports full, stage1, and stage2. |
--diagnostics | Optional modality diagnostics for prediction. | none / mcr / mir / full |
{quickstartPrediction}{quickstartReasoning}For API models such as GPT-4.1 and Gemini-3-Flash-Preview, export API keys before running evaluation.
{quickstartApiKeys}Results on VT-Bench across two task families. For discriminative prediction, classification reports Accuracy and regression reports RMSE. For generative reasoning, all entries report exact-match accuracy.
Classification metrics are higher-is-better; regression metrics are lower-is-better. Δ denotes the relative performance gap to the best unimodal model.
| Classification (↑) | Regression (↓) | |
|---|---|---|
| {header} | )}||
| {row.label} | ||
| {cell} | )}
The table includes open-source general models, proprietary models, and tool-augmented methods.
| Open-Source General Models | Proprietary Models | Tool-Augmented Methods | |
|---|---|---|---|
| {header} | )}|||
| {row.label} | |||
| {cell} | )}