Ê׸ö²âÊÔʱ¹²½ø»¯ºÏ³É¿ò¼ÜTTCS£ºÔÚ¡¸×óÓÒ»¥²«¡¹ÖÐÍ»ÆÆÍÆÀíÆ¿¾±
2026-02-26 08:52:05

ÔÚ DeepSeek-R1 ºÍ OpenAI o1 ÒýÁìµÄ¡¸ºóѵÁ·£¨Post-Training£©¡¹Ó롸²âÊÔʱÀ©Õ¹¡¹£¨Test-Time Scaling£©¡¹Ê±´ú£¬£¬ £¬ £¬£¬£¬ÔõÑùʹÓòâÊÔʱµÄËãÁ¦¾ÙÐÐÓÐÓÃѵÁ·³ÉΪ½¹µã¡£¡£¡£¡£¡£¡£¡£¡£

È»¶ø£¬£¬ £¬ £¬£¬£¬ÃæÁÙ¼«ÄѵIJâÊÔÌ⣬£¬ £¬ £¬£¬£¬ÏÖÓеIJâÊÔʱѵÁ·£¨Test-Time Training, TTT£©ÍùÍùÒòα±êÇ©ÔëÉù´ó¶øÏÝÈ롸Ϲ²Â¡¹µÄÄæ¾³¡£¡£¡£¡£¡£¡£¡£¡£

ÏÃÃÅ´óѧ DeepLIT ¿ÎÌâ×éÌá³öÁËÒ»ÖÖȫеIJâÊÔʱ¿Î³ÌºÏ³É¿ò¼Ü ¡ª¡ªTTCS (Test-Time Curriculum Synthesis)¡£¡£¡£¡£¡£¡£¡£¡£¸Ã¿ò¼Ü²»ÒÀÀµÈκÎÍⲿÈ˹¤±ê×¢£¬£¬ £¬ £¬£¬£¬Í¨¹ýÌìÉúÆ÷£¨Synthesizer£©ÓëÇó½âÆ÷£¨Solver£©µÄ¹²½ø»¯²©ÞÄ£¬£¬ £¬ £¬£¬£¬×Ô¶¯ºÏ³É´¦ÓÚÄ£×Ó¡¸ÄÜÁ¦½çÏß¡¹µÄ¿Î³ÌÊý¾Ý£¬£¬ £¬ £¬£¬£¬½â¾öÁ˲âÊÔÑù±¾¹ýÄѵ¼ÖµÄѵÁ·Ì®ËúÎÊÌâ¡£¡£¡£¡£¡£¡£¡£¡£

ʵÑéÏÔʾ£¬£¬ £¬ £¬£¬£¬ÔÚ Qwen2.5-Math-1.5B ÉÏ£¬£¬ £¬ £¬£¬£¬TTCS ½«ÊýÑ§ÍÆÀíÆ½¾ù·Ö´Ó 17.30 ±©ÕÇÖÁ 41.49£¬£¬ £¬ £¬£¬£¬ÔÚ¼«¾ßÌôÕ½µÄ AIME ¾ºÈüÌâÉϸüÊÇÏÔÖøÓâÔ½ÁË TTRL µÈÇ¿»ùÏß¡£¡£¡£¡£¡£¡£¡£¡£

ÂÛÎÄÎÊÌ⣺TTCS: Test-Time Curriculum Synthesis for Self-EvolvingÂÛÎÄÁ´½Ó£ºhttps://arxiv.org/abs/2601.22628ÏîÄ¿´úÂ룺https://github.com/XMUDeepLIT/TTCSHuggingFace Ö÷Ò³£ºhttps://huggingface.co/papers/2601.22628

01. ½¹µãÄîÍ·£ºµ±¡¸Ì⺣սÊõ¡¹Óöµ½¡¸³¬¸ÙÄÑÌ⡹

Ëæ×Å´óÓïÑÔÄ£×Ó£¨LLM£©µÄÉú³¤£¬£¬ £¬ £¬£¬£¬Òµ½ç¹²Ê¶ÒÑ´Ó´¿´âµÄ¡¸Ô¤ÑµÁ·À©´ó²ÎÊý¡¹×ªÏòÍÚ¾ò²âÊÔʱÀ©Õ¹£¨Test-Time Scaling£©µÄDZÁ¦¡£¡£¡£¡£¡£¡£¡£¡£

DeepSeek-R1 µÈÄ£×ÓµÄÀÖ³É֤ʵÎúÇ¿»¯Ñ§Ï°£¨RL£©ÔÚÍÆÀíʹÃüÉϵÄÖØ´óÍþÁ¦¡£¡£¡£¡£¡£¡£¡£¡£È»¶ø£¬£¬ £¬ £¬£¬£¬ÏÖÔÚµÄ RL ·¶Ê½£¨Èç RLVR£©Í¨³£ÒÀÀµ´ó×Ú¸ßÖÊÁ¿µÄ Ground Truth ±êÇ©£¬£¬ £¬ £¬£¬£¬ÕâÏÞÖÆÁËÆäÀ©Õ¹ÐÔ¡£¡£¡£¡£¡£¡£¡£¡£

ΪÁËÕõÍѶԱêÇ©µÄÒÀÀµ£¬£¬ £¬ £¬£¬£¬²âÊÔʱѵÁ·£¨Test-Time Training, TTT£©Ó¦Ô˶øÉú¡£¡£¡£¡£¡£¡£¡£¡£Æä½¹µãÍ·ÄÔÊÇ£ºÔÚ²âÊԽ׶Σ¬£¬ £¬ £¬£¬£¬Ê¹ÓÃÄ£×Ó¶Ô²âÊÔÌâÌìÉúµÄα±êÇ©£¨Pseudo-labels£©¾ÙÐм´Ê±¸üС£¡£¡£¡£¡£¡£¡£¡£

µ«ÔÚÃæÁÙ¸ßÄѶÈÍÆÀíʹÃü£¨Èç AIME Êýѧ¾ºÈü£©Ê±£¬£¬ £¬ £¬£¬£¬TTT ÔâÓöÁËÖÂÃüµÄ¡¸ÄÜÁ¦´íÅ䡹ã£ÂÛ£º

α±êÇ©²»¿É¿¿£¨Noisy Rewards£©£ºÃæÁÙ AIME ÕâÖÖÄÑÌ⣬£¬ £¬ £¬£¬£¬ÓÉÓÚÄ£×Ó×Ô¼ºÄÜÁ¦È±·¦£¬£¬ £¬ £¬£¬£¬×ÝÈ»¾ÙÐжà´Î²ÉÑù£¨Majority Voting£©£¬£¬ £¬ £¬£¬£¬´ó²¿·ÖÃÕµ×Ò²ÊÇ´íµÄ¡£¡£¡£¡£¡£¡£¡£¡£»£»£» £»£»ùÓÚ¹ýʧµÄ¹²Ê¶¾ÙÐÐѵÁ·£¬£¬ £¬ £¬£¬£¬Ö»»áÈÃÄ£×Ó¡¸Ô½Ñ§Ô½Æ«¡¹¡£¡£¡£¡£¡£¡£¡£¡££¨Èç Intro ÖÐ Figure 1 (a) Ëùʾ£©È±·¦ÖÐÐÄõè¾¶£¨Steep Learning Curve£©£º¾ÍÏñÈÃСѧÉúÖ±½Ó×ö΢»ý·Ö£¬£¬ £¬ £¬£¬£¬È±·¦ÖÐÐÄÄѶȵĹý¶ÉÎÊÌ⣬£¬ £¬ £¬£¬£¬Ä£×ÓÎÞ·¨¿çÔ½ÖØ´óµÄÄѶȺ蹵£¬£¬ £¬ £¬£¬£¬µ¼ÖÂÓÅ»¯Ê§°Ü¡£¡£¡£¡£¡£¡£¡£¡£

ÍŶÓ˼Ë÷£ºÈôÊÇûÓÐÏÈÉú£¬£¬ £¬ £¬£¬£¬Ä£×ÓÄÜ·ñÏñÈËÀà×ÔѧÕßÒ»Ñù£¬£¬ £¬ £¬£¬£¬Í¨¹ý×Ô¼º¸ø×Ô¼º³öÌ⣬£¬ £¬ £¬£¬£¬°ÑÒ»µÀÄÑÌâ²ð½â±äΪһ×é³ÉÌݶȡ¢¿É½â¾öµÄѵÁ·Ì⣬£¬ £¬ £¬£¬£¬´Ó¶øÊµÏÖÂÝÐýÉÏÉý £¿£¿£¿ £¿£¿

»ùÓÚ´Ë£¬£¬ £¬ £¬£¬£¬ÍŶÓÌá³öÁËTTCS¡ª¡ª Ò»¸ö»ùÓÚ¹²½ø»¯£¨Co-evolution£©µÄ²âÊÔʱ¿Î³ÌºÏ³É¿ò¼Ü¡£¡£¡£¡£¡£¡£¡£¡£

02. ÒªÁìÂÛ£ºSynthesizer Óë Solver µÄË«ÖØ¹²½ø»¯

TTCS µÄ½¹µãÔÚÓÚ¹¹½¨ÁËÒ»¸ö±Õ»·µÄÉú̬ϵͳ£¬£¬ £¬ £¬£¬£¬°üÀ¨Á½¸ö¹²Ïí³õÊ¼È¨ÖØµÄ Agent£¬£¬ £¬ £¬£¬£¬ËüÃÇͨ¹ýµü´úʽµÄGRPO (Group Relative Policy Optimization)¾ÙÐÐѵÁ·£º

½ÇÉ«·Ö¹¤

Òªº¦»úÖÆ£ºÔõÑùѰÕÒ¡¸ÄÜÁ¦½çÏß¡¹ £¿£¿£¿ £¿£¿

ΪÁ˱ÜÃâÌìÉúµÄÎÊÌâÌ«¼òÆÓ£¨ÎÞЧѵÁ·£©»òÌ«ÄÑ£¨ÎÞЧ·´Ï죡£¡£¡£¡£¡£¡£¡£©£¬£¬ £¬ £¬£¬£¬ÍŶÓÉè¼ÆÁËÒ»Ì×ϸÃܵÄÄÜÁ¦×Ô˳Ӧ½±Àø£¨Capability-Adaptive Reward£©»úÖÆ£º

±Õ»·Ð§Ó¦£º

Solver ÄÜÁ¦ÌáÉýºó£¬£¬ £¬ £¬£¬£¬Æä¡¸ÄÜÁ¦½çÏß¡¹»áÏò¸üÄѵÄÇøÓòÒÆ¶¯£¬£¬ £¬ £¬£¬£¬¸ÐÖªµ½Õâһת±äµÄ Synthesizer ΪÁË»ñµÃ¸ß½±Àø»á×Ô¶¯ÌìÉúÄѶȸü¸ßµÄÎÊÌ⣬£¬ £¬ £¬£¬£¬Solver ¼ÌÐø¹¥¿ËÐÂÄÑÌâ¡£¡£¡£¡£¡£¡£¡£¡£¶þÕßÏ໥׷¸Ï£¬£¬ £¬ £¬£¬£¬ÔÚ²âÊÔʱʵÏÖÁ˶¯Ì¬µÄ Curriculum Learning¡£¡£¡£¡£¡£¡£¡£¡£

03. Ó²ºËʵÑéЧ¹û£ºÍÆÀíÄÜÁ¦ÌáÉý

ÍŶÓÔÚ AMC23¡¢AIME24¡¢AIME25¡¢MATH-500¡¢Minerva µÈ¶à¸öȨÍþÊýѧ»ù×¼ÉϾÙÐÐÁËÆÕ±éÑéÖ¤¡£¡£¡£¡£¡£¡£¡£¡£»£»£» £»£»ù´¡Ä£×Óº­¸Ç Qwen2.5-Math (1.5B, 7B) ºÍ Qwen3-4B¡£¡£¡£¡£¡£¡£¡£¡£

1. Ö÷Á÷»ù×¼ÉϵÄÄëѹʽÌáÉý

TTCS ÔÚËùÓгߴçµÄÄ£×ÓÉϾùÈ¡µÃÁËÏÔÖøÓÅÓÚ»ùÏßµÄЧ¹û£¨Ïê¼û Table 1£©£º

Qwen2.5-Math-1.5B£ºTTCS µÄƽ¾ù·ÖµÖ´ï£¬£¬ £¬ £¬£¬£¬Ïà±ÈԤѵÁ·Ä£×Ó£¨17.30£©ÌáÉýÁË+24.19 ·Ö£¡×ÝÈ»±ÈÕÕ²âÊÔʱ΢µ÷µÄÇ¿»ùÏß TTRL£¨36.56£©£¬£¬ £¬ £¬£¬£¬ÒÀÈ»Óдó·ùÓÅÊÆ¡£¡£¡£¡£¡£¡£¡£¡£Qwen2.5-Math-7B£ºTTCS ƽ¾ù·ÖµÖ´ï£¬£¬ £¬ £¬£¬£¬Ïà±ÈÆÕ±éʹÓÃµÄ Self-Consistency£¨32.15£©ÌáÉýÁË+20.39 ·Ö£¬£¬ £¬ £¬£¬£¬ÔÙ´Î֤ʵÎú×Ô¶¯µÄ²âÊÔʱѵÁ·Ô¶Ê¤ÓÚ±»¶¯µÄ¶à·²ÉÑù¡£¡£¡£¡£¡£¡£¡£¡£

2. ¹¥¿Ë¸ßÄѶȾºÈüÌâ (AIME)

ÔÚ×îÄÜÌåÏÖÍÆÀíÉÏÏÞµÄ AIME ¾ºÈüÌâÉÏ£¬£¬ £¬ £¬£¬£¬TTCS Õ¹ÏÖÁ˼«Ç¿µÄ¹¥¼áÄÜÁ¦£º

AIME 2024£ºÔÚ 1.5B Ä£×ÓÉÏ£¬£¬ £¬ £¬£¬£¬TTRL ½öµÃ 13.23 ·Ö£¬£¬ £¬ £¬£¬£¬¶ø TTCS Ô¾ÉýÖÁ·Ö£¬£¬ £¬ £¬£¬£¬ÌáÉý·ù¶È´ïAIME 2025£ºÔÚ 7B Ä£×ÓÉÏ£¬£¬ £¬ £¬£¬£¬TTCS µÖ´ïÁË·Ö£¬£¬ £¬ £¬£¬£¬ÏÔÖøÓÅÓÚ TTRL µÄ 14.06 ·Ö¡£¡£¡£¡£¡£¡£¡£¡£ÕâÓÐÁ¦µØÖ¤ÊµÎú£ºµ±²âÊÔÌâÌ«Äѵ¼Ö TTRL µÄα±êǩʧЧʱ£¬£¬ £¬ £¬£¬£¬TTCS ͨ¹ýºÏ³ÉÖÐÐÄÄѶȵĿγÌ£¬£¬ £¬ £¬£¬£¬ÀֳɼܯðÁËͨÍù¸ßÄѶÈÍÆÀíµÄÇÅÁº¡£¡£¡£¡£¡£¡£¡£¡£

3. Ϊʲô TTCS ÓÐÓà £¿£¿£¿ £¿£¿£¨Éî¶ÈÆÊÎö£©

ΪÁË̽ÌÖÐÔÄÜÌáÉýµÄȪԴ£¬£¬ £¬ £¬£¬£¬ÍŶӾÙÐÐÁ˶àά¶ÈµÄÆÊÎö£º

²»µ«½öÊÇÊýѧ£¨·º»¯ÐÔ£©

ʵÑé Q1 ÏÔʾ£¬£¬ £¬ £¬£¬£¬ÔÚ AIME ÉϾÙÐвâÊÔʱѵÁ·µÄ TTCS Ä£×Ó£¬£¬ £¬ £¬£¬£¬ÔÚMMLU-ProºÍSuperGPQAµÈͨÓÃÁìÓòÍÆÀíʹÃüÉÏҲʵÏÖÁËÐÔÄÜÔ¾Éý£¨Figure 3a£©¡£¡£¡£¡£¡£¡£¡£¡£Õâ˵Ã÷Ä£×Óѧµ½µÄÊÇͨÓõÄÍÆÀíÂß¼­£¬£¬ £¬ £¬£¬£¬¶ø·Ç¼òÆÓµÄ¹ýÄâºÏ¡£¡£¡£¡£¡£¡£¡£¡£

¶¯Ì¬ÏÈÉú > ¾²Ì¬Ãûʦ

ÍŶÓʵÑéÓÃÒ»¸ö¸üÇ¿µ«Àο¿µÄQwen2.5-14B-InstructÄ£×Ó×÷Ϊ Synthesizer£¨ÊµÑé Q3£©¡£¡£¡£¡£¡£¡£¡£¡£Ð§¹ûÁîÈ˾ªÑÈ£º¹²½ø»¯µÄ 1.5B Synthesizer (TTCS) ´øÀ´µÄÌáÉý (+5.34) ¾¹È»ÊǾ²Ì¬ 14B Synthesizer (+2.66) µÄÁ½±¶£¡

ÕâÕ¹ÏÖÁËÒ»¸öÉî¿ÌµÄÔ­Àí£ºË³Ó¦Ñ§ÉúÄ¿½ñˮƽµÄÏÈÉú£¬£¬ £¬ £¬£¬£¬±È´¿´âˮƽ¸ßµ«²»¶®Òò²ÄÊ©½ÌµÄÏÈÉú¸üÖ÷Òª¡£¡£¡£¡£¡£¡£¡£¡£

Êý¾ÝЧÂʾªÈË

×ÝȻֻʹÓà 10% µÄ²âÊÔÊý¾Ý£¨½ö 3 µÀÌ⣩£¬£¬ £¬ £¬£¬£¬TTCS ÔÚ AIME24 ÉϵÄ׼ȷÂʾÍÄִܵï13.33£¬£¬ £¬ £¬£¬£¬Ô¶³¬Ò»ÂÉÊý¾ÝÁ¿Ï TTRL µÄÌåÏÖ£¨Figure 4£©¡£¡£¡£¡£¡£¡£¡£¡£ÕâÅú×¢ TTCS Äܹ»¸ßЧµØÕ¥È¡Ã¿Ò»¸ö²âÊÔÑù±¾µÄÐÅÏ¢Á¿¡£¡£¡£¡£¡£¡£¡£¡£

04. ×ܽáÓëÕ¹Íû

TTCS ¿ò¼ÜµÄÌá³ö£¬£¬ £¬ £¬£¬£¬ÊǶԡ¸²âÊÔʱÅÌË㡹·¶Ê½µÄÒ»´ÎÖØ¹¹¡£¡£¡£¡£¡£¡£¡£¡£ÍŶÓ֤ʵÎú£ºÔÚ²âÊÔʱ£¬£¬ £¬ £¬£¬£¬Ä£×Ó²»Ó¦ÊDZ»¶¯µÄ½âÌâÕߣ¬£¬ £¬ £¬£¬£¬¶øÓ¦ÊÇ×Ô¶¯µÄ¿Î³ÌÉè¼ÆÕß¡£¡£¡£¡£¡£¡£¡£¡£

ͨ¹ý Synthesizer ºÍ Solver µÄ¹²½ø»¯£¬£¬ £¬ £¬£¬£¬TTCS ÍêÉÆ½â¾öÁ˲âÊÔʱѵÁ·ÖС¸Êý¾Ýϡȱ¡¹ºÍ¡¸ÄѶȶϲ㡹µÄÁ½´óÍ´µã¡£¡£¡£¡£¡£¡£¡£¡£

Õâ²»µ«ÎªÌáÉý´óÄ£×ÓÔÚÖØ´óÊýѧÌì½ò±õº£ÐÂÄÜÔ´ÓÐÏÞ¹«Ë¾¡¢´úÂëÍÆÀíʹÃüÉϵÄÌåÏÖÌṩÁËÐÂ˼Ð÷£¬£¬ £¬ £¬£¬£¬Ò²ÎªÎ´À´×Ô½ø»¯ÖÇÄÜÌ壨Self-Evolving Agents£©¡ª¡ª ¼´Ä£×ÓÄܹ»ÔÚÍêȫδ֪µÄÇéÐÎÖÐͨ¹ý×ÔÎÒ²©ÞÄʵÏÖÒ»Á¬½ø»¯ ¡ª¡ª µÓÚ¨Á˼áʵ»ù´¡¡£¡£¡£¡£¡£¡£¡£¡£