Source-linked AI summary
MBA: Multimodal Benchmark and Agents for Real-World Business Ideation
Hojun Choi, Jaeyo Shin, Suin Lee, Hyunjung Shim
TL;DR
Existing business-ideation agents largely rely on text despite real-world contexts containing visual information that captions may omit. This paper introduces MBA-Bench and two specialized multimodal agents, which outperform caption-based and multimodal baselines across the benchmark.
Problem
Existing business-ideation approaches are text-dependent, although visual details can contain business-relevant information that text cannot fully capture.
Method
MBA-Bench pairs multimodal samples across six domains with business questions and evidence-grounded reference ideas, while MBA-b and MBA-k optimize setting-specific rewards.
Results
MBA-b outperforms caption-based and multimodal baselines by 63.9% and 25.6%, while MBA-k does so by 77.1% and 35.8%, respectively.
Takeaways & Limitations
The agents consistently outperform diverse baselines while remaining competitive with strong closed-source multimodal large language models.
Takeaways & Limitations
MBA currently focuses on image and text, excluding informative audio, olfactory, tactile, and other sensory signals.
Abstract
from arXiv · showhide
Agentic systems powered by large language models (LLMs) have opened new opportunities for business ideation. Yet existing approaches remain confined to a text-only paradigm, despite the inherently multimodal nature of real-world contexts. We thus introduce MBA-Bench, the first multimodal benchmark for training and evaluating business ideation agents, comprising 30K samples across six domains, each domain characterized by distinct visual cues not fully conveyed by text alone. Concretely, we automatically caption images and employ GPT-4o to generate five reference ideas for each of three business questions through retrieval query generation, market evidence retrieval, and evidence-augmented synthesis. Following prior work, we evaluate agents across six business-oriented criteria using MLLM-as-a-Judge. To consider settings where criteria are hidden or disclosed, we present MBA-b and MBA-k for blind and known, respectively. We train both with two novel reward objectives---creativity and feasibility---while MBA-k further optimizes the six disclosed criteria for eight in total. Both are trained via LoRA-based supervised fine-tuning followed by group relative policy optimization with these setting-specific rewards. For extensive experiments on MBA-Bench, we set up two baselines accommodating either captions only or multimodal inputs, with the latter nearing closed-source performance on several metrics. MBA-b and MBA-k outperform caption baselines by 63.9% and 77.1%, and multimodal baselines by 25.6% and 35.8%, respectively.
1 Introduction
MBA reframes business ideation as a multimodal task because text-only methods miss visual information and multimodal prompting remains limited in creativity. It introduces MBA-Bench and two specialized agents that improve over caption and multimodal baselines across blind and known evaluation settings.
- Motivation: Venture capital firms evaluate roughly 100 deals per completed investment, motivating automated support for iterative idea generation and screening.Each deal requires months of due diligence.
- Motivation: Existing ideation methods rely on a text-in, text-out paradigm, while images contain unique information that text cannot fully capture.Captioning can omit details in overly complex scenes, creating a modality gap.
- Benchmark: MBA-Bench is the first multimodal benchmark for business ideation agents, comprising 30K multimodal samples across six domains and eight business-oriented dimensions.Its domains include General, Spatial Layout, Crowding, Visual Condition, Shape & Texture, and Technical Features.
- Agents: MBA-b and MBA-k target blind and known evaluation settings, respectively, optimizing creativity and feasibility, with MBA-k additionally using disclosed evaluation criteria.Both variants are trained with supervised fine-tuning and group relative policy optimization under setting-specific objectives.
- Results: 63.9% and 25.6%: MBA-b outperforms caption-based and multimodal baselines, while MBA-k achieves 77.1% and 35.8% improvements, respectively.MBA-k remains competitive with closed-source MLLMs.
2 Related Work
Prior work extends LLMs with planning and tool use for multi-step reasoning, applies these capabilities to entrepreneurship and patent-based commercialization, and develops reward-based post-training methods for alignment and reasoning.
- Agentic entrepreneurship: Agentic AI frameworks augment LLMs with planning and tool use for multi-step reasoning.This text-based foundation has been extended to entrepreneurship, where systems generate ideas grounded in market needs and originality.
- Agentic entrepreneurship: Patent documents provide a rich source of technologies intended for commercialization in entrepreneurship-focused AI research.PBIG introduces six business-oriented dimensions for judging patent-derived proposals.
- Reinforcement learning: RL supports LLM alignment and reasoning through methods including PPO, DPO, and GRPO.GRPO retains a scalar reward while removing the value network and estimating advantages comparatively.
3 Methodology
MBA reframes business ideation as a multimodal task through MBA-Bench, a 30K-sample benchmark spanning six domains and grounded in retrieved market evidence. It further introduces blind and known agents trained with setting-specific rewards and evaluated by an MLLM-as-a-Judge.
- Task-specific agents: MBA-b and MBA-k address blind and known evaluation settings, respectively, with both sharing creativity and feasibility rewards while MBA-k also uses six disclosed criteria.Creativity measures novelty relative to five reference ideas, whereas feasibility captures market relevance and factual consistency.
- Multimodal scope: The benchmark targets visual business cues that text alone cannot fully convey, including both easily verbalized and difficult-to-verbalize content.Its six domains include General, Spatial Layout, Crowding, Visual Condition, Shape & Texture, and Technical Features.
- MBA-Bench construction: MBA-Bench comprises 30K image–caption–question–idea quadruplets across six domains, curated through GPT-4o and DuckDuckGo.Each image is paired with a caption, three business questions, and five reference ideas generated through visual query extraction, market evidence retrieval, and evidence-augmented ideation.
- Agent training: Training combines LoRA-based supervised fine-tuning with group relative policy optimization using setting-specific ranking rewards.GRPO steers both agents beyond homogeneous priors toward creative, feasible ideas, while MBA-k additionally incorporates the six disclosed dimensions.
- Evidence-grounded ideation: The ideation protocol grounds reference ideas in market evidence retrieved from visual observations rather than model speculation.GPT-4o derives retrieval queries from salient image content, which are submitted to the DuckDuckGo API for relevant evidence such as existing solutions and wait-time statistics.
- Evaluation: An MLLM-as-a-Judge evaluates generated ideas across six business-oriented dimensions using a strong 78B-parameter MLLM.Separate MLLM judges are used for training and evaluation to avoid bias from reusing the same judge.
4 Experiments
Experiments characterize MBA-Bench, compare MBA agents with multimodal and caption-only systems, and analyze performance across domains, modalities, and training objectives. Results show strong domain-consistent performance, advantages in differentiated and market-oriented ideation, robust reward generalization, and persistent modality mismatch in caption-based reconstruction.
- Benchmark and setup: MBA-Bench contains 30K image–caption–question–idea quadruplets across six domains, with 15 reference ideas per image and unified prompts containing four supporting elements.Each image has a caption, three business questions, and five reference ideas per question; prompts include domain, query, question, and evidence.
- Overall results: MBA-k shows the clearest advantages in Competitive Advantage and Market Size while remaining competitive with the strongest proprietary models in Innovativeness.Figure 5 averages 15 responses for each of 100 test images and normalizes six metrics to a common 0–100 scale.
- Domain analysis: MBA-k achieves near-perfect, highly consistent Specificity across domains, while MBA-b follows a similarly uniform but slightly lower pattern.Both agents score especially highly in Technical Features and Visual Condition for Technical Validity, while Innovativeness and Competitive Advantage also perform strongly in Spatial Layout.
- Modality analysis: Caption-only systems often lack critical visual cues and perform poorly across domains, whereas even accurate captions cannot reconstruct unverbalizable visual semantics.The caption baseline is an exception in General, where simpler everyday scenes are more easily verbalized; qualitative examples include misplaced signage and a swirling 3D pattern.
- Training objectives: 51.2% and 114.3% are the agents’ respective leads over the multimodal baseline across the two reward-oriented metrics.Spearman’s ρ values of 0.83 and 0.71 across model groups indicate that creativity and feasibility rewards generalize across rubrics for original and viable ideation.
5 Limitations & Future Work
MBA establishes a foundation for broader multimodal business ideation but identifies three limitations that hinder effective real-world deployment: incomplete sensory coverage, absent temporal reasoning, and insufficient personalization to entrepreneurs.
- Future work: These limitations define future research directions for advancing MBA toward more effective real-world business ideation.The framework is intended to extend beyond patent-constrained ideation toward broader multimodal settings.
- Multimodal coverage: MBA currently uses image and text, leaving informative audio, olfactory, tactile, and other sensory signals unexplored.These modalities may reveal latent needs and opportunities, including emotion, regional characteristics, and domain-specific cues.
- Temporal reasoning: MBA does not explicitly model temporal information, although video could provide motion, behavioral, and causal context absent from single frames.A video may reveal safety risks and motivate preventive solutions that a static image would suggest as parking-related services.
- Entrepreneur personalization: MBA generates and evaluates ideas independently of the prospective entrepreneur, so feasibility cannot reflect user-specific capital, expertise, location, networks, or risk tolerance.Future work should condition both generation and evaluation on personalized information, despite the difficulty of collecting and analyzing it.
6 Conclusion
The paper introduces MBA-Bench, a 30K-sample multimodal benchmark for business ideation, and two task-specialized agents trained for blind and known evaluation settings. Both agents outperform diverse baselines while remaining competitive with strong closed-source MLLMs.
- Contributions: MBA-Bench is the first multimodal benchmark for business ideation, comprising 30K multimodal samples across six domains with up to eight metrics.The benchmark is designed to support multimodal business-ideation evaluation.
- Contributions: The paper proposes two task-specialized agents for blind and known evaluation settings using SFT–GRPO training.The agents are specialized to the two evaluation configurations.
- Results: Both agents consistently outperform diverse baselines while remaining competitive with strong closed-source MLLMs.The conclusion reports consistent gains over baselines alongside competitive performance against closed-source models.
A Experimental Details · A.1 Device Information
Experiments ran on a fixed Ubuntu 22.04 LTS software and hardware setup using eight NVIDIA RTX A6000 GPUs and dual AMD EPYC 7513 CPUs. Training and evaluation used approximately 45 GiB and 30 GiB of GPU memory per device, respectively.
- A.1 Device Information: Experiments used Ubuntu 22.04 LTS, eight NVIDIA RTX A6000 GPUs, and dual AMD EPYC 7513 CPUs.This was the hardware and operating-system configuration for all experiments.
- A.1 Device Information: Training used approximately 45 GiB of GPU memory per device, while evaluation used approximately 30 GiB.The reported memory allocation differed between training and evaluation.
- A.1 Device Information: The software environment comprised Python 3.10.13, PyTorch 2.2.0 with CUDA 12.1, and random seed 2026.These versions and the seed were used throughout the experiments.
A.2 Implementation Details … MBA-Bench Evaluation
The paper constructs MBA-Bench from visually grounded images, captions, retrieval evidence, and reference ideas, then trains MBA agents with LoRA-based SFT followed by GRPO. Evaluation uses unified multimodal prompts and an MLLM judge across six business-oriented dimensions.
- A.2 Implementation Details: MBA agents start from Qwen2.5-VL-7B-Instruct and use LoRA-based supervised fine-tuning followed by GRPO, with Qwen2.5-VL-72B-Instruct as reward judge and InternVL2.5-78B for evaluation.PaliGemma2 captions images and GPT-4o constructs MBA-Bench.
- A.3 Evaluation Details: The evaluation judge scores each generated idea across six dimensions with ranges 1–4, 1–4, 1–5, 1–4, 0–3, and 0–3.The shared multimodal inputs and unified question prompt follow the PBIG evaluation protocol.
- A.4 Hyperparameters: MBA-k weights eight rewards, while MBA-b uses creativity and feasibility weights of 0.70 and 0.30.MBA-k’s eight rewards include six disclosed criteria plus creativity and feasibility; GRPO uses four sampled responses, a KL coefficient of 0.02, and batch size 4.
- A.5 Pseudocode: The construction algorithm selects representative images, captions them, extracts retrieval queries, retrieves market evidence, generates K reference ideas, and splits data by image at 95:5.The pipeline combines image, caption, domain, query, and evidence for each business question.
- MBA-Bench Construction: 30,000 samples result from 2,000 images, three business questions, and five reference ideas per question, split by image to prevent overlap.The construction spans domain-specific image selection, visually grounded retrieval, and evidence-augmented prompting.
- MBA-Agent Training: MBA-b optimizes creativity and feasibility, while MBA-k additionally optimizes the six disclosed evaluation metrics using group-relative rewards.Training first produces an SFT policy, then GRPO samples candidate ideas and computes creativity relative to references and feasibility from market relevance and factuality.
- MBA-Bench Evaluation: Each model generates one idea per unified test prompt, and evaluation averages 15 instance-level scores per image before computing means and standard deviations over 100 image-level results.The score vector covers Specificity, Technical Validity, Innovativeness, Competitive Advantage, Need Validity, and Market Size.
B Dataset Details … C Additional Quantitative Evaluation
MBA-Bench combines visual evidence, domain context, captions, business questions, retrieval queries, and market evidence to ground multimodal business ideation. Its datasets span six visually distinct domains, while MBA-Library supports feasibility through scholarly, structured, semantic-search, and factuality evidence evaluated with paired statistical comparisons.
- B.1 MBA-Bench: MBA-Bench integrates images, domains, captions, business questions, retrieval queries, and retrieved market evidence into each ideation sample.Images provide primary visual evidence, while retrieval grounds ideas in industry practices, customer needs, and market facts.
- ADE20K: 500 ADE20K images form the General domain, providing diverse, comparatively caption-verbalizable indoor and outdoor contexts.Images are ranked by annotated object-category diversity and semantic richness.
- RICO: 350 RICO screenshots represent Spatial Layout, emphasizing dense controls, menus, text, and component relationships difficult to infer from isolated descriptions.The domain supports ideation involving interface usability, service design, and user engagement.
- MS-COCO: 350 MS-COCO images constitute the Crowding domain, selected for scenes containing at least ten annotated people and diverse objects.These scenes support ideas about customer flow, capacity, safety, accessibility, and resource allocation.
- VisA: VisA and DeepPCB supply visually implicit industrial domains: Visual Condition and Technical Features, with 350 VisA images and 100 PCB images.They target defects, surface irregularities, components, traces, and manufacturing diagnostics for quality inspection, maintenance, and reliability applications.
- C Additional Quantitative Evaluation: The additional evaluation compares MBA-k with GPT-5 mini, Gemini 3.1 Pro, and InternVL2.5-8B using paired two-sided Wilcoxon tests on 100 images.Each image contributes averaged responses from 15 outputs, producing 100 paired image-level observations and reporting Δ in favor of MBA-k when positive.
- B.2 MBA-Library: MBA-Library grounds feasibility rewards in external evidence by integrating scientific literature, structured entities, and Wikipedia-based knowledge.OpenAlex and Wikidata provide complementary technical and commercial coverage, while FAISS and FActScore support retrieval and factuality verification.
- FAISS Library: FAISS retrieves top-k MBA-Library records and normalizes idea–evidence similarity to [0, 1] as a market-relevance feasibility component.FActScore decomposes ideas into atomic claims, verifies them against retrieved Wikipedia passages, and normalizes support to [0, 1].
C.1 Further Statistical Analysis · D Additional Ablation Studies · D.1 Additional Captioning Models
MBA-k shows statistically significant gains in most model–metric comparisons, while captioner choice has limited downstream impact. Captioners perform broadly comparably, with modest model-specific advantages across evaluation dimensions.
- C.1 Further Statistical Analysis: 12 of 18 model–metric comparisons show statistically significant improvements for MBA-k.The result is reported in Table A2.
- C.1 Further Statistical Analysis: MBA-k significantly outperforms all three comparators in Innovativeness, Competitive Advantage, and Market Size.
- C.1 Further Statistical Analysis: MBA-k improves Need Validity over GPT-5 mini and InternVL2.5-8B and Specificity over InternVL2.5-8B.
- C.1 Further Statistical Analysis: GPT-5 mini performs better on Specificity and Technical Validity, while Gemini 3.1 Pro performs better on Specificity, Technical Validity, and Need Validity.The passage reports these as converse comparisons against MBA-k.
- D Additional Ablation Studies: Replacing PaliGemma2 with Gemini or GPT-5 while holding the ideation model and remaining inputs fixed tests captioner effects on downstream performance.The ablation changes only the image captioner.
- D.1 Additional Captioning Models: The three captioners achieve broadly comparable results across the six evaluation dimensions.The comparison appears in Table A3.
- D.1 Additional Captioning Models: Gemini and GPT-5 provide modest gains in Specificity and Technical Validity, whereas PaliGemma2 remains competitive in Innovativeness.
D.2 Reliability and Failure Analysis
MBA-k exhibits the strongest reliability among open-source models, with the lowest severe semantic-failure rate and substantially fewer invalid-format outputs than other open-source baselines. Its remaining format violations primarily reflect incomplete or nonconforming four-field JSON rather than inability to generate meaningful ideas.
- Failure definitions: Severe semantic failure means rubric scores ≤2 for Specificity, Technical Validity, Innovativeness, or Competitive Advantage, or ≤1 for Need Validity or Market Size.The Technical rate is the subset attributable to Technical Validity.
- Invalid formats: 3.40% is MBA-k’s invalid-format rate, compared with 16.93–99.80% for other open-source models.These violations primarily involve incomplete or nonconforming four-field JSON outputs and do not necessarily indicate an inability to generate meaningful ideas.
E Additional Qualitative Results … Prompt for Evaluation
The appendix provides representative MBA-Bench instances across six visual domains and documents the prompts used for benchmark construction, supervised fine-tuning, policy optimization, and evaluation. These prompts preserve image-grounded context and structured business-idea generation while specifying stage-specific inputs and outputs.
- E Additional Qualitative Results: Representative samples span six visual domains and show each complete instance from image–caption pair through domain, question, query, retrieved evidence, and generated idea.Evaluation scores for each metric appear in the upper-right corner of each generated idea box.
- Prompt for MBA-Bench: The benchmark-construction prompt combines the input image, target user question, business perspective, and retrieved market and technical evidence to generate five concise, visually grounded reference ideas.These ideas serve as expert trajectories for subsequent training stages.
- Prompt for SFT: The SFT prompt combines the image, domain context, auxiliary caption, image-specific annotations, business lens, and paired market evidence to produce one idea in a four-field JSON format.The prompt is designed for grounded business-idea generation from multimodal and retrieved evidence.
- Prompt for GRPO: The GRPO prompt retains the SFT stage’s canonical input structure so policy optimization changes idea quality without changing the task formulation or output schema.This preserves consistency between supervised fine-tuning and policy rollouts.
- Prompt for Evaluation: The evaluation prompt applies identical image-grounded context and structured JSON requirements across models, enabling controlled comparison under the same generation setting.The task instructs models to use the visible image as primary visual evidence and generate exactly one grounded business idea for the specified question–query pair.
- Prompt for Evaluation: Evaluation examples cover General, Spatial Layout, Crowding, Visual Condition, Shape & Texture, and Technical Features domains, pairing visual scenes with business queries and generated ideas.Examples include mobile-app automation, quality inspection, and other image-grounded business contexts.
- Prompt for Evaluation: The evaluation setup consistently uses one Q<QUERY_INDEX>/W<QUERY_INDEX> pair and requires exactly one grounded business idea from the visible image and paired context.The same task instruction is shown for evaluation, SFT, and GRPO prompt templates.