MACM: Utilizing a Multi-Agent System for Condition Mining in Solving Complex Mathematical Problems

• MACM targets insufficient complex-mathematical reasoning and inadequate generalizability in existing prompting methods.Prior methods include Tree of Thought and Graph of Thought, but require problem-specific prompts.Source: §1 · pp. 1–2
• MACM abstracts each problem’s conditions and objective, then uses a multi-agent system to iteratively mine new conditions until the objective can be reached.Source: Figure 1 · p. 1, §1 · p. 2
• The Conditions-and-Objective formulation removes the need to manually design prompts for each problem and continually expands known conditions toward the final answer.The method compresses information from multiple thoughts into existing conditions rather than relying on hierarchical traversal and filtering.Source: §1 · p. 2
• Across the MATH dataset, 24-point game, and number-sequence sorting, MACM showed generalizability and stronger error correction than original prompting methods.MATH accuracy increased by 15.14%, improved by 7.8% over SC-CoT, and exceeded ToT by 17% on the 24-point game.Source: §1 · p. 2

• I-O prompting directly supplies problem conditions and generates answers through token-level sequential decisions.Source: §2 · p. 2
• CoT prompting constructs explicit reasoning chains to improve logical progression and coherence in complex problem solving.Source: §2 · p. 2
• SC-CoT extends CoT by having large models vote or score outputs and selecting the most coherent response.Source: §2 · p. 2
• ToT organizes ideas hierarchically in a tree and uses voting or scoring to refine structured reasoning and reduce errors.Source: §2 · pp. 2–3
• GoT connects thoughts across tree branches, decomposes complex tasks into subtasks, and incrementally merges their outcomes.Source: §2 · p. 3

3.1. MACM Overall Structure

• Three agents divide the workflow: Thinker generates ideas, Judge evaluates correctness and sufficiency, and Executor performs the final calculations.The Judge adds verified conditions and discards incorrect ones before the Executor applies the approved steps.Source: §3.1 · p. 3
• MACM initializes a Condition List and Objective, repeatedly mines and judges new conditions, then plans and executes a solution once conditions suffice.The process allows up to five mining iterations before treating an unmet objective as unsolvable.Source: §3.1 · p. 3

3.2. Using Cases

• In the algebra case, the Judge rejects an initially incorrect Thinker condition, after which verified conditions support the Executor’s numerical calculation.The rejected condition would otherwise have entered the Known Conditions list and affected downstream code.Source: §3.2 · pp. 3–4
• In the geometry case, MACM mines and verifies successive conditions before planning and executing the calculation, whereas the original GPT-4 Turbo produced an incorrect result from faulty relational expressions.The example includes verification of a right-triangle condition and a subsequent equation before solving.Source: §3.2 · p. 4
• MACM extracts each problem’s conditions and objective before prompting, replacing problem-specific prompt design with the generalized form Evaluate {objective} {input}.The paper contrasts this with manually configuring ToT prompts for individual tasks.Source: §3.2 · p. 4

3.3. Multi-Agents System

• The detailed process applies one prompt set to different problem types, with examples covering algebra and geometry and stages for extracting conditions, mining new conditions, and solving.The examples use GPT-4 Turbo in the Thinker, Judge, and Executor roles.Source: §3.2 · p. 3, Figure 3 · p. 5, §3.3 · p. 5
• MACM assigns distinct identities and response formats to its agents, producing consistent outputs that facilitate Judge evaluation and new-condition extraction.The paper presents Thinker, Judge, and Executor instructions and links response standardization to automatic system operation.Source: §3.3 · p. 6

4.1. Performance on MATH benchmark

• On the full MATH dataset, MACM increased GPT-4 Turbo’s accuracy by 20% over the original model and by 7.8% over SC-CoT, while geometry remained weaker.Geometry improved by 17.63% but reached only 62.74% accuracy, whereas algebra and number theory reached 96.07% and 98.04%, respectively.Source: §4.1 · pp. 6–7
• 31.82% and 31.92% accuracy gains occurred on Level 5 Geometry and Intermediate Algebra, respectively, with improvements across all seven categories.The largest gains occurred in categories where original GPT-4 Turbo performed worst: Geometry at 18.18% and Intermediate Algebra at 34.04%.Source: §4.1 · p. 7, Figure 5 · p. 7

4.2. Comparison with GoT and ToT

• 17% higher accuracy than ToT was achieved by MACM with GPT-4 on the 24-point game, while its Judge corrected mistakenly evaluated thoughts.The comparison used standard GPT-4 without code capabilities and matched ToT at search breadth b = 5.Source: §4.2 · p. 7, Table 2 · p. 7
• 2.94% higher sorting accuracy than GoT was reported for MACM on 64-number sequence sorting with GPT-3.5.The GoT result was estimated from its graph because specific data were unavailable.Source: Table 3 · p. 7, §4.2 · p. 8

4.3. Trade-off Between Accuracy and LLM Queries

• Nearly 90% of GPT-4 Turbo’s original errors were corrected by MACM at high query counts, whereas I-O, CoT, and SC-CoT corrected only about 20%.MACM’s multiple condition-mining and checking processes let the model gradually identify errors as responses increased.Source: §4.3 · p. 8, Figure 6 · p. 8

5. Conclusion

• MACM improved GPT-4 Turbo accuracy by 15.14% on MATH, 7.8% over SC-CoT, from 54.68% to 76.73% on Level 5 problems, and 17% over ToT on 24-point.The experiments are described as demonstrating generalizability and error-correction capability for complex mathematical problems.Source: §5 · p. 8

6. Limitation and Discussion

• MACM’s practical limitations are increased problem-solving time from multiple invocations and weaker performance on geometry problems.The paper proposes future work using MACM-assisted corrected datasets to refine the model’s intrinsic capabilities.Source: §6 · p. 8

A. Why is the generalizability of ToT and GoT limited

• ToT requires specially designed prompts for each experiment and task, including separate proposal, value, and final-step prompts for the 24-point game.Its documentation also states that adding a new task requires task-specific prompt setup.Source: A. Why is the generalizability of ToT… · p. 10
• GoT likewise uses problem-specific instructions, such as a sorting prompt tailored to splitting 32 numbers into two lists.The paper reports this requirement across GoT tasks including sorting, set operations, keyword counting, and document merging.Source: A. Why is the generalizability of ToT… · p. 10
• MACM is presented as addressing this generalizability challenge created by tailoring prompts to each specific problem.The contrast is between task-specific prompt engineering in ToT and GoT and MACM’s broader applicability.Source: A. Why is the generalizability of ToT… · p. 10

MATH dataset

• MATH evaluation used GPT-4 Turbo on randomly selected general and difficulty-level-5 subsets, comparing I-O, CoT, SC-CoT, and MACM with standardized sampling settings.The tests used one-third of the undifferentiated MATH dataset and half of the questions from each level-5 category; all methods used n = 1, Topk = 1, and temperature t = 0.7.Source: B. Experimental Setup · p. 10

24-point game

• The 24-point evaluation tested 100 relatively difficult games ranked 901–1,000 from 4nums.com using GPT-4 and GPT-3.5.Success required producing a valid equation equaling 24 while using each input number exactly once; MACM used n = 1, Topk = 1, temperature t = 0.7, and thinker, judge, and executor token limits of 512, 4, and 256.Source: B. Experimental Setup · p. 10

Sequence sorting

• Sequence sorting was evaluated on 100 randomly generated sequences containing 64 elements with GPT-3.5 and the same MACM sampling configuration.The configuration used n = 1, Topk = 1, temperature t = 0.7, and thinker, judge, and executor token limits of 512, 4, and 256.Source: B. Experimental Setup · p. 10