yd2333Ôƶ¥µç×ÓÓÎÏ·

½Ó´ýÀ´µ½ÉϺ£yd2333Ôƶ¥µç×ÓÓÎÏ·ÔƲÖ,½ß³ÏΪÄúÌṩ:µçÉִ̲¢ÅäËÍ,ÀäÁ´²Ö´¢ÅäËÍ,»õ²ÖÍâ°üµÈ²ÖÅäÒ»Ì廯ЧÀÍ!ÊÕ²ØÎÒÃÇ µçÉÌÔÆ²Ö ÍøÕ¾µØͼ

½Ó´ýÀ´µ½ÉϺ£yd2333Ôƶ¥µç×ÓÓÎÏ·ÔƲÖ,½ß³ÏΪÄúÌṩ:µçÉִ̲¢ÅäËÍ,ÀäÁ´²Ö´¢ÅäËÍ,»õ²ÖÍâ°üµÈ²ÖÅäÒ»Ì廯ЧÀÍ!

È«ÎÂÇøʳƷ²ÖÅäÎïÁ÷ЧÀÍÉÌ20Íò©O×ÔÓª²Ö+AAAA¼¶ÎïÁ÷+È«¹úÀäÁ´ÎïÁ÷°ÙÇ¿

182-0218-6162400-096-2966

yd2333Ôƶ¥µç×ÓÓÎÏ·¶¯Ì¬

ÈÈËÑÒªº¦´Ê£º µçÉÌÒ»¼þ´ú·¢ ÀäÁ´ÅäËÍ ÉçÇøÍŹº²ÖÅä ʳƷ²Ö´¢´ú·¢»õ ÔöֵЧÀÍ

»ùÓÚ¸ÅÂʲ¢Ðмƻ®µÄ×Ô¶¯ÎïÁ÷²Ö´¢½¨Ä£Óëµ÷Àí

À´Ô´£º±¾Õ¾ | Ðû²¼ÈÕÆÚ£º2023-03-02

0 ÒýÑÔ

Ëæ×ÅÎïÁ÷¶©µ¥´ÓÆ·ÖÖÉÙ¡¢ÅúÁ¿´óµ½Æ·ÖֶࡢÅúÁ¿Ð¡»òÅú´Î¶à¡¢ÅúÁ¿Ð¡µÄÉú³¤£¬²Ö´¢¹¦Ð§ÒÑ´Ó¿¼ÂǴ洢ЧÂʵÄÖØÒªÐÔÖð½¥±äΪ¿¼ÂÇÈçºÎºÏÀíµØ·ÖÅɲÖλÒÔ¼°Ë³ÀûµØ×Ô¶¯´æ·¢»õ¡£ÖÇÄִܲ¢ÏµÍ³±»¹ã·ºÓ¦ÓÃÓÚ²Ö´¢ºÍÎïÁ÷µÄ¸÷¸ö·½Ã棬ºÏÀí¸ßЧµÄ²Ö´¢µ÷Àí²»µ«´ó´ó¼õÇáÁËÊÂÇéÈËÔ±µÄÀͶ¯Ç¿¶È£¬²¢ÇÒÔÚÌá¸ß²Ö´¢¿Õ¼äÀûÓÃÂʺͽµµÍÈËÁ¦±¾Ç®µÈ·½Ãæ¾ßÓÐÃ÷ÏÔµÄÓÅÊÆ¡£ÖÇÄִܲ¢µ÷ÀíϵͳµÄ½¨Ä£ÓëÇó½âÒѾ­³ÉΪҵÄÚÈËÊ¿µÄÈÈÃÅÑо¿ÁìÓò¡£

¹Å°åµÄÎïÁ÷²Ö´¢³µ¼äºÍÔË×÷ÌåϵÒÀÀµ´ó×ÚµÄÈËÊÂÇéÒµ£¬ÐèÒª´ó×ÚµÄÈËÁ¦±¾Ç®¡£²¢ÇҹŰåµÄ¿âλ·ÖÅÉ·½·¨±£´æ×Ųִ¢¿âλ¼°½ø³ö¿â»õÎïÊý¾Ý¸üлºÂý¡¢¿âλÓÐЧÀûÓÃÂÊÆ«µÍ¡¢¿âλ·ÖÅÉģʽ¾²Ì¬»¯¡¢²Ö´¢ÄÚ°áÔËЧÂʵÍϵÈÎÊÌ⣬´Ó¶ø±¬·¢¸ß¶îµÄ¿â´æ±¾Ç®¡£Ä¿Ç°£¬ÔÚÕë¶ÔÎïÁ÷²Ö´¢µ÷ÀíÓÅ»¯¼¼ÊõºÍÒªÁìÖУ¬½¨Ä£¼°Çó½âÒѾ­³ÉΪ¼ìÑé¸Ã²Ö´¢µ÷Àíϵͳ¼°¾ö²ßÊÇ·ñ¸ßЧµÄÖ÷Òª¼¼ÊõÖ®Ò»¡£ÔÚÉè¼ÆеĿâλÓÅ»¯ÒªÁì»òÔÚ¶ÔÔËÊäÉ豸µÄÐж¯ÓÅ»¯Ê±£¬ÀûÓÃÒ»¸öÁ¼ºÃµÄ²Ö´¢µ÷ÀíÄ£ÐͽøÐÐÄ£ÄâÇó½âÄÜÌṩֱ¹ÛµÄ¾ö²ßÖ§³Ö¡£

¹ØÓÚÖÇÄִܲ¢µ÷ÀíÎÊÌâµÄ½¨Ä£ÓëÇó½â£¬ÏÖÓеÄÐí¶àÒªÁìÒÀÀµÓÚ»ùÓÚÌض¨Ä£Ð͵ÄÓÅ»¯ÒªÁ죨¾ßÌå¼ûµÚ1Õ£©¡£ÕâЩģÐͲÎÊý¶à¡¢È˹¤¸ÉÔ¤¶à£¬³£ÐèÔÚʵÑéÀú³ÌÖÐÊÖ¶¯½ç˵¶à¸ö²ÎÊýÖµ¡£²ÎÕÕijÎïÁ÷¹«Ë¾ÌṩµÄ»õ²Ö¿âλ¼°¶Ñ¶â»úÐÅÏ¢£¬±¾ÎÄÌá³öÒ»ÖÖ»ùÓÚ¸ÅÂʲ¢Ðмƻ®µÄÎïÁ÷×Ô¶¯²Ö´¢µ÷ÀíÒªÁ졣ƾ¾Ý²Ö´¢¿âλ¼°ÖÇÄÜÌå´æÈ¡»õÐж¯µÄ²»È·¶¨ÐÔ¼°²¢ÐÐÐÔµÈÌص㣬¸ÃÒªÁìÊ×Ïȹ¹½¨Ò»¸öÐÎʽ»¯µÄ¼Æ»®ÁìÓòÃèÊöÄ£ÐÍ£¬È»ºóʹÓøÅÂʲ¢Ðмƻ®¼¼ÊõÀ´½øÐÐ×Ô¶¯Çó½â¡£±¾ÎÄÖ÷ÒªÀûÓÃÖÇÄܼƻ®ÒªÁì¶Ô×Ô¶¯²Ö´¢µ÷ÀíÁìÓò½øÐн¨Ä££¬È»ºóÀûÓüƻ®Çó½â¼¼Êõ½øÐÐ×Ô¶¯Ä£Äâ¡£¹¹½¨µÄ¼Æ»®ÁìÓòÄ£ÐÍ°üÀ¨Á˶ÔÖÖÖÖÓ°ÏìÒòËØ¡¢Ô¼ÊøÌõ¼þÒÔ¼°¿ÉÄÜʼþµÄÐÎʽ»¯ÃèÊö£¬ÒÔÇó½â»ñµÃ¿É±íÕ÷Õû¸öµ÷ÀíÀú³ÌµÄÐж¯Õ½ÂÔ¡£Õû¸öÇó½âÀú³Ì²»ÐèÒªÈ˹¤¸ÉÔ¤£¬¿É½âÊÍÐÔÇ¿£¬Ö±¹ÛÒ׶®¡£

1 Ñо¿Åä¾°

1.1 ÎïÁ÷×Ô¶¯²Ö´¢

Åãͬ×ŵç×ÓÉÌÎñ¡¢È˹¤ÖÇÄÜ¡¢ÐÂÄÜÔ´µÈÐÐÒµµÄ¿ìËÙÉú³¤£¬ÎïÁ÷¶©µ¥ÌØÕ÷·ºÆðÆ·ÖֶࡢÖÜÆڶ̡¢ÅúÁ¿ÉÙ¡¢Åú´Î¶àµÄÇ÷ÊÆ¡£È»¶øÔÚµ±½ñµÄÆóÒµÓÃÊÂÇéÒµ´óÇé¿öÖУ¬ÈË¿ÚºìÀûÖð½¥¼õÈõ£¬ÍÁµØºÍÀͶ¯Á¦±¾Ç®²»¾øÉÏÉý¡£ÒÔÀͶ¯Ãܼ¯ÐÍΪÖ÷ÒªÐÎʽµÄ¹Å°å²Ö´¢ÎïÁ÷ÒµÎñÐÎʽÒѾ­²»¿ÉÂú×ãÄ¿½ñ½×¶ÎµÄʵ¼ÊÒµÎñÐèÇó¡£²Ö´¢ÎïÁ÷¼¼ÊõµÄÉú³¤¼±ÐèתÐÍÉý¼¶ÒÔʵÏÖÖÇÄÜ»¯¡£

½üÄêÀ´£¬º£ÄÚÍ⳧É̶¼ÔÚ»ý¼«¹Ø×¢²¢Á¢Ò쿪·¢ÖÇÄÜÎïÁ÷²Ö´¢ÏµÍ³¡£ÑÇÂíÑ·ÂÊÏÈÀÖ³ÉÑо¿ÁËKiva Systems»õ²Ö»úеÈËϵͳ£¬Ê¹ÓÃÊý°Ù¸ö×Ô¶¯µ¼º½³µ£¨automated guided vehicle,AGV£©È¡´úÁËÊÂÇéÈËÔ±Íê³É»õÎﶩµ¥µÄ´æÈ¡ÈÎÎñ£¬Ê¹ÊÂÇéÈËÔ±ÃâÓÚÿÌìÂíÀ­ËÉʽµØÀ´»Ø´æÈ¡»õÎï²¢¼«´óµØÌá¸ßÁ˲ÖÖü´æÈ¡²Ù×÷µÄЧÂÊ[1]¡£º£ÄÚ³§ÉÌÈ缫ÖǼΡ¢º£¿µµÈÏȺóʵÏÖÁËÀàËÆKivaÖÇÄÜϵͳµÄÏȽø¼¼Êõ²¢Õ¼¾ÝÁ˴󲿷ÖÊг¡·Ý¶î£¬¾©¶«Ò²ÕýÔÚÑз¢½¨ÉèÍêÈ«ÎÞÈ˵Ä×Ô¶¯»¯ÎïÁ÷ÖÐÐÄ¡£ÔÚ¼¼Êõ²»¾øÁ¢ÒìÓëÈںϵÄÅä¾°Ï£¬×Ô¶¯²Ö´¢AGVµÄ¹¦Ð§¼°ÐÔÄÜ»ñµÃ¼«´óµÄ¸ïкÍÌáÉý£¬¶øʵÏÖ¶©µ¥ÓëAGVµÄºÏÀíµ÷ÀíÕ½ÂԲŻª¾ö¶¨Õû¸ö×Ô¶¯ÎïÁ÷²Ö´¢ÏµÍ³µÄÐÔÄÜ¡£

Õë¶ÔÖÇÄִܲ¢µ÷ÀíÎÊÌâµÄ½¨Ä£ÓëÓÅ»¯£¬½üÄêÀ´º£ÄÚÍâÐí¶àѧÕß½øÐÐÁËÉîÈëµÄ̽ÌÖ¡£AoyamaµÈÈË[2]»ùÓÚµç×ÓÉÌÎñµÄÉú³¤Ç÷ÊƼ°ÎïÁ÷¹¤ÒµµÄģʽ£¬Ê¹ÓÃÃÉÌØ¿¨ÂÞ·ÂÕæÉè¼ÆÁËÎïÁ÷ÒÔ¼°ÐÅÏ¢Á÷Á½¸öÄÚ²¿Á¬½ÓÍøÂçµÄ×îÓÅ»¯Ä£ÐÍ£¬Í¨¹ýʵÑé·ÂÕæÄ£ÄâÐÅÏ¢ºÍÔËÊäÓöÈÒòËØÔÚÎïÁ÷ÆóÒµÖеÄÓ°Ïì¡£Ballest¨ªnµÈÈË[3]¿¼ÂÇÁËÔÚ¾ßÓÐËæ»ú´æ´¢ºÍ²î±ðÀàÐͲ泵µÄʵ¼Ê»õ²ÖÖд洢ºÍ¼ìË÷ÎÊÌ⣬Ϊÿ¸ö´æ´¢»ò¼ìË÷µÄλÖÃÑ¡Ôñ×ã¹»ÊìÁ·µÄ²æ³µ£¬²¢È·¶¨Ã¿¸ö²æ³µÖ´ÐвÙ×÷µÄ˳Ðò£¬ÒÔʹ×ܵÄʹÓÃʱ¼ä×îС»¯£¬Í¨¹ý½«ÎÊÌâÆÊÎöΪÈý¸ö×ÓÎÊÌâ²¢»ùÓÚÆô·¢Ê½Ëã·¨½â¾ö¡£ChengµÈÈË[4]¸ïÐÂÁË´æ´¢/ÔËÊäÎïÁ÷ϵͳµÄ×îÓÅ»¯ÎÊÌ⣬»ùÓÚÀëɢʼþ½¨Ä£ºÍËæ»úÓÅ»¯¿ØÖÆÉè¼ÆÁËÒ»ÖÖÓÃÓÚ¾ö²ßÖ§³ÖÎïÁ÷ϵͳµÄ½¨Ä£·ÂÕæËã·¨£¬Ê¹¾ö²ßÕßÁ˽⶯̬Ëæ»úϵͳµÄÐÐΪ¡£º£ÄÚѧÕßÓÚºÕÄê[5]ÒÔ²Ö´¢¿Õ¼äÇé¿öÄ£ÐÍΪÑо¿¹¤¾ß£¬ÆÊÎö¶àÖÇÄÜÌåµ÷ÀíϵͳµÄÈÎÎñ·ÖÅÉģʽ¡¢×÷ÒµÁ÷³Ì¼°ÈÎÎñµ÷ÀíÔ¼ÊøÌõ¼þ£¬ÒÔ×îС»¯×î´óÈÎÎñÍê³Éʱ¼äΪÖ÷ҪĿ±ê£¬½¨Á¢¸´ºÏÖ¸±êµÄÊýѧģÐÍÈ·¶¨AGVµÄµ÷Àí¼Æ»®¡£Ó¡ÃÀµÈÈË[6]ÒԶѶâ»úÐÐʻʱ¼äºÍ»õ¼ÜÎȶ¨ÐÔΪÓÅ»¯Ä¿±ê£¬»ùÓÚParetoÒÅ´«Ëã·¨½¨Á¢»õλ¶àÄ¿±êÓÅ»¯µÄÊýѧģÐÍÇóµÃ¶¯Ì¬»õλÓÅ»¯×îӎ⣬½Ï´ó¸ÄÉÆÊÕÖ§¿âЧÂʼ°ÄÜÁ¿ÏûºÄ¡£Â³½¨ÏõÈÈË[7]̫ͨ¹ýÎö¿ç²ã´©Ëó³µ²Ö´¢ÏµÍ³ÖÐÌáÉý»úÓë´©Ëó³µµÄʵ¼Ê×÷ÒµÁ÷³Ì£¬½¨Á¢¸´ÏàÖúҵ·¾¶¼Æ»®Ä£ÐͲ¢»ùÓÚÈ˹¤ÓãȺËã·¨Çó½â¸ÃÄ£ÐÍ£¬ÓÐЧµØÌá¸ß¸Ã²Ö´¢ÏµÍ³µÄÔËÐÐЧÂÊ¡£

1.2 ¸ÅÂʲ¢Ðмƻ®

ÖÇÄܼƻ®[8]ÊÇÈ˹¤ÖÇÄܵÄÒ»¸öÖØÒª·ÖÖ§¡£Æä¼Æ»®Àú³ÌÊÇÑ°ÕÒ´Ó³õʼ״̬µ½Ä¿±ê״̬µÄÒ»¸öÎÞ³åÍ»¿ÉÓ¦ÓõÄÐж¯ÐòÁС£¸ÅÂʲ¢Ðмƻ®ÊǸÅÂʼƻ®ºÍ²¢Ðмƻ®µÄ½áºÏÌ壬ÔÊÐí·Ç»¥³âµÄÐж¯²¢·¢Ö´ÐУ¬Ðж¯µÄÇ°ÌáºÍЧ¹ûÂú×ãÈ«¾ÖÐÔÔ¼Êø£¬ËÉ¿ªÁ˾­µä¼Æ»®µÄÎå´ó¼ÙÉè[9]¡£ÓÉÓÚÔÚ¸ÃÀàÎÊÌâÖÐÐж¯¾ßÓв»È·¶¨ÐԺͲ¢ÐÐÐÔ£¬ËùÒÔÔÚÄ£ÄâÏÖʵÎÊÌâÉÏÓкܴóµÄÓ¦ÓÃDZÁ¦¡£ÈĶ«ÄþµÈÈË[10]Õë¶Ô¹ÉƱÊг¡¾ßÓдó×ڵIJ»È·¶¨ÐԺͲ¢·¢ÐÔ£¬»ùÓÚ¸ÅÂʲ¢Ðмƻ®¹¹½¨¹ÉָģÄâÎÊÌâµÄ¼Æ»®ÁìÓò²¢Çó½â»ñµÃ¹ÉƱģÄâÖ¸Êý¡£¿ÉÊÇ£¬ÓÉÓÚ²»È·¶¨ÐԺͲ¢ÐÐÐԵĽáºÏ£¬×´Ì¬ËÑË÷¿Õ¼äÔÚ×î³õµÄ¼¸¸öËÑË÷°ì·¨ÖбäµÃºÜÊÇ´ó£¬ËùÒԹŰåµÄËÑË÷Õ½ÂÔºÍÆô·¢Ê½ÆÀ¹ÀЧÂʺܵÍ£¬ÈçºÎ¸üºÃ¸ü¿ìµØÇó½â¸ÅÂʲ¢Ðмƻ®ÎÊÌâÒ²ÊǽüÄêÀ´¼Æ»®ÁìÓòµÄÈÈÃÅÑо¿Æ«Ïò[11,12]¡£

¸ÅÂʲ¢Ðмƻ®ÎÊÌâÓɼƻ®ÓïÑÔ¹Øϵ¶¯Ì¬Ó°ÏìͼÓïÑÔRDDLÀ´½øÐÐÃèÊöµÄ¡£RDDLÊÇ»ùÓÚ¹æÔòµÄ¼Æ»®ÃèÊöÓïÑÔ£¬ÆäÓïÒåΪ¹Øϵ¶¯Ì¬±´Ò¶Ë¹ÍøÂ磨relational dynamic Bayesian network,RDBN)[13]¡£RDDLÊǾ­¹ý֮ǰµÄ¼Æ»®ÓïÑÔÑݽø¹ýÀ´µÄ£¬¾ßÓÐÐí¶àÆäËû¼Æ»®ÓïÑÔËù²»¾ß±¸µÄÐÂÌØÐÔ£¬°üÀ¨Ò»Çк¬ÓвÎÊýµÄ·ÇÁ÷£¨non-fluent£©»òÁ÷£¨fluent£©µÄ±äÔª¡£ËüÃèÊöÁ˺¯Êý±ä¸ï×÷Ϊ״̬תÒÆ¡¢Í¨¹ýÂß¼­±í´ïʽ»òÂþÑܺ¯ÊýÀ´ÃèÊö»Ø±¨º¯Êý¡¢£¨²¿·Ö¿ÉÊӲ죩Âí¶û¿É·ò¾ö²ßÀú³Ì£¨partially observable Markov decision process,(PO)MDP£©Ä¿±êºÍ¶ÔÆäµÄ״̬/Ðж¯Ô¼ÊøµÈÌØÐÔ¡£RDDLʹÓÃÂí¶û¿É·ò¾ö²ßÀú³Ì£¨MDP)[14]À´½¨Ä£¾ßÓÐ˳Ðò¾ö²ßµÄÎÊÌ⣬ÆäÖÐϵͳÔÚʱ¼äÉϲ»¾øÉú³¤£¬²¢ÇÒÓÉÊðÀí¿ØÖÆ¡£Âí¶û¿É·ò¾ö²ßÀú³Ì£¨Markov decision processes,MDP£©ÃèÊöÁËÒ»¸ö¶¯Ì¬Àú³Ì£¬Ã¿¸ö¼Æ»®²½ÊðÀíÊÓ²ìÇé¿ö£¬ËæºóËüÑ¡ÔñÖ´ÐÐÒ»¸öÐж¯£¬Ðж¯±»Ö´ÐкóÇé¿ö±¬·¢¸Ä±ä£¬ËæºóÊðÀí»á»ñµÃÇé¿ö±ä¸ïºóµÄ½±Àø¡£MDP½ç˵ÈçÏ£ºM=(S,A£¬¦Õ£¬R£©£¬ÆäÖÐSºÍA»®·ÖÊÇÓÐÏÞµÄ״̬ºÍÐж¯¼¯£»¦Õ£¨s,a,s¡ä£©ÊÇÔÚ״̬sÖ´ÐÐÐж¯aºóÇé¿öÑÝ»¯µ½s¡äµÄ¸ÅÂÊ£»RÊǻر¨º¯Êý£¬R(a,s£©½ç˵ÁËÔÚ״̬sÖ´ÐÐÐж¯a»ñµÃµÄÖм佱Àø¡£MDPµÄ½â½Ð×öÕ½ÂÔ£¨policy£©£¬¼ò³Æ¦Ð£¬½ç˵ÁËÔÚÿ¸ö״̬ÊðÀíÒªÖ´ÐеÄÐж¯¡£ÓÃV¦Ð£¨s£©ÌåÏÖÕ½ÂÔ¦ÐÏÂ״̬sµÄÖµº¯Êý£¬×îÓÅÕ½ÂÔ¦Ð*ÊÇÖ¸»ñµÃÀÛ¼ÆÆô·¢½±Àø×î´óµÄÕ½ÂÔV*£¬Âú×ã±´¶ûÂüµÈʽ¡£

yd2333Ôƶ¥µç×ÓÓÎÏ·ÓÎÏ·(Öйú)ÓÐÏÞ¹«Ë¾ 


ÆäÖУº¦Ã£¨0¡Ü¦Ã¡Ü1£©ÎªÕÛ¿ÛϵÊý£¬Ëü±»ÓÃÀ´¶ÔδÀ´µÄÐж¯½øÐÐÕÛ¿Û½±Àø£¬ÒòΪ²»È·¶¨ÐÔ¾ö¶¨ÁËÕâ¸öÀú³Ì¡£

2 ÎÊÌâÃèÊöÓë¼ÙÉè

²ÎÕÕijÎïÁ÷¹«Ë¾ÌṩµÄ²Ö´¢¿âλ¼°¶Ñ¶â»úÐÅÏ¢£¬±¾ÎÄÄ£ÐÍʹÓÃRDDL¼Æ»®ÓïÑÔ½øÐн¨Ä£¡£»õ²ÖÄ£ÐÍÈçͼ1Ëùʾ¡£

ͼ1 ²Ö´¢Ä£ÐÍ

ͼ1 ²Ö´¢Ä£ÐÍ   


½«Õû¸ö»õ²ÖµÄ¹¤¾ßÁýͳΪһ¸öÈýάÌåϵ£¨Í¼1(a£©£©£¬ÌåϵÖаüÀ¨ÓÐÖÇÄÜÌ壨¶Ñ¶â»ú£©¡¢»õ¹ñ¡¢²î±ðλÖõĻõÎï¡¢¶à¸öͨµÀ¡¢Ò»¸öÈë»õ¿ÚºÍ³ö»õ¿Ú£¨Í¼1(b£©£©¡£¶Ñ¶â»úÖ»ÄÜÔÚͨµÀÀïÒƶ¯²¢ÇÒƾ¾Ý¶©µ¥ÐèÇó½øÐдæ»õºÍÈ¡»õ¡£¶Ñ¶â»úµÄÐж¯ÊÇÈ·¶¨ÐԵģ¬ÔÚÈë»õ¿Ú»òÕß³ö»õ¿Ú»®·ÖÊÇÈë»õºÍ³ö»õÐж¯£¬ÔÚͨµÀÖÐÊÇÒƶ¯Ðж¯£¬µÖ´ï´æ´¢»õÎïµÄ»õ¹ñλÖÃʱ´æ·Å»òÄÃÈ¡»õÎһ̨¶Ñ¶â»úµÄ´æ·ÅÈÝÁ¿ÊÇÓÐÏÞÖƵÄ£¬¼´Ò»Ì¨¶Ñ¶â»ú×Ô¼ºÖ»ÄÜ´æ·ÅÒ»¶¨µÄ»õÎï¡£»õ¹ñ¼´´æ·Å»õÎïµÄµØ·½£¬Í¨µÀ¡¢Èë»õ¿ÚºÍ³ö»õ¿ÚµÄλÖö¼ÊÇÀι̵Ä¡£¼ÙÉè»õÎïµÄ´æ·ÅλÖÃÊÇÓÉϵͳƾ¾Ý¶©µ¥×´Ì¬Ëæ»úµÃ³öµÄ£¬²¢ÇÒÖ»Óлõ¹ñΪ¿ÕºÍ»õ¹ñµÄ¾ÞϸÇкϻõÎï¾ÞϸµÄʱºò¶Ñ¶â»ú²Å»ª´æ·Å»õÎÁíÍâ»õ¹ñÊÇÁ½²ãµÄ£¬ÄÚÍâ²ã¶¼ÊÇ¿ÉÒÔ´æ·Å»õÎïµÄ¡£ÓÉÓÚ»õÎïµÄ´æ·ÅλÖÃÊDzî±ðµÄ£¬ËùÒÔÈë»õµ½´æ»õ¡¢È¡»õµ½³ö»õʱÿ̨¶Ñ¶â»úµÄÒƶ¯Ò²ÊDzî±ðµÄ£¬×îÖÕÍê³ÉÄ¿±êµÄʱ¼äÒ²ÊDzî±ðµÄ¡£

3 Ä£Ð͹¹½¨

±¾ÎIJִ¢µ÷ÀíµÄRDDLÁìÓòÄ£ÐÍÓÉÁù¸ö²¿·Ö×é³É¡£RDDLÎļþ°üÀ¨ÁËÁìÓò£¨domains£©ºÍʵÀý£¨instances£©Á½ÖÖÀàÐ͵Ķ¥¼¶ÉùÃ÷¡£RDDLÁìÓòÓÉÁù¸ö²¿·Ö×é³É£¬°üÀ¨requirements¡¢types¡¢pvariables¡¢cpfs¡¢rewardºÍconstraints¡£Ã¿¸ö²¿·ÖÔÚ¹¹½¨ÍêÕûµÄ¸ÅÂʲ¢Ðмƻ®ÁìÓòÀú³ÌÖж¼ÊDz»¿Éȱ°±ÉÄ¡£ÊµÀýÄ£¿é°üÀ¨ÁË·ÇÁ÷±äÁ¿¡¢ÁìÓòÎÊÌ⹤¾ß¡¢ÁìÓòµÄ³õʼ״̬¡¢¿É²¢·¢±äÁ¿ºÍÕÛ¿ÛÒò×ӵȡ£Õë¶Ô±¾ÎIJִ¢µ÷ÀíµÄ¾ßÌ幤¾ß£¬ÁìÓòÄ£ÐÍ£¨https£º//github.com/KimkaychanJK/warehouse£ßscheduling£ßdomain£©¹¹½¨Àú³ÌÈçÏ£º

a)requirements¡£¸Ã²¿·ÖÃèÊö¸÷¸öÁìÓò¾ßÌå²Ù×÷µÄÐèÇ󣬰üÀ¨ÁìÓòµÄÌصãºÍËùÐèÒªµÄÔ¼ÊøµÈ¡£±¾ÎÄʹÓÃÁËÈýÖÖÐèÇó£¬reward-deterministic˵Ã÷ÁìÓòÊÇʹÓÃ×Ô½ç˵µÄÌض¨»Ø±¨Öµ£»constrained-state˵Ã÷ÁìÓòʹÓÃ״̬ԼÊø£»concurrent˵Ã÷ÁìÓò¹¤¾ß¿ÉͬʱִÐжà¸öÐж¯¡£¾ßÌåΪ

yd2333Ôƶ¥µç×ÓÓÎÏ·ÓÎÏ·(Öйú)ÓÐÏÞ¹«Ë¾



b)types¡£¸Ã²¿·Ö½ç˵¸Ã²Ö´¢µ÷ÀíÁìÓòµÄ¹¤¾ß£¬»®·ÖΪstacker£¨¶Ñ¶â»ú£©¡¢bin£¨»õ¹ñ£©¡¢aisle£¨Í¨µÀ£©£¬x£ßsection¡¢y£ßshelf¡¢z£ßlevel»®·Ö´ú±í»õ²ÖÈýάÌåϵµÄÈý¸öÆ«Ïò¡£¾ßÌåΪ

yd2333Ôƶ¥µç×ÓÓÎÏ·ÓÎÏ·(Öйú)ÓÐÏÞ¹«Ë¾



c)pvariables²¿·Ö¡£¸Ã²¿·Ö½ç˵ӰÏì¸Ã²Ö´¢µ÷ÀíµÄ±äÁ¿ÀàÐÍ£¨»®·ÖÓзÇÁ÷ÎÄ×Ö£¨non-fluent£©¡¢Á÷ÎÄ×Ö£¨fluent£©¡¢Ðж¯Á÷£¨action-fluent£©¡¢×´Ì¬Á÷£¨state-fluent£©¡¢ÖмäÁ÷£¨interm-fluent£©ºÍÊÓ²ìÁ÷£¨observfluent£©£©£¬²¢½ç¶¨Ã¿ÖÖ±äÁ¿µÄ±ä¸ï¹æÄ£¡£±¾²Ö´¢µ÷ÀíÁìÓòʹÓÃÁËnon-fluent¡¢state-fluentºÍaction-fluent¡£non-fluentÓëfluentʾÀýÈçÏ£º

yd2333Ôƶ¥µç×ÓÓÎÏ·ÓÎÏ·(Öйú)ÓÐÏÞ¹«Ë¾



non-fluentÌåÏÖÔÚÕû¸ö¼Æ»®Àú³ÌÖмá³ÖÎȶ¨µÄÁ¿£¬ËüÖ»»á¸Ä±äÿ¸öÎÊÌâµÄ½ç˵¡£±¾ÎÄÁìÓòÖеÄnon-fluent»®·ÖΪadjacent-forwardºÍadjacent-backward£¨¶Ñ¶â»úÒƶ¯Æ«Ïò£©¡¢binAt£¨»õ¹ñλÖã©¡¢aisleAt£¨Í¨µÀλÖã©¡¢inboundItemAt£¨Èë»õ¶©µ¥Î»Öã©¡¢inboundBase£¨Èë»õ¿ÚλÖã©¡¢outboundBase£¨³ö»õ¿ÚλÖ㩺Íoutbound£ßprob£¨»õÎï³ö»õµÄ¸ÅÂÊ£©¡£

state-fluent±äÁ¿ÌåÏÖÔڼƻ®Àú³ÌÖбä¸ïµÄ״̬£¬»®·ÖΪstackerAt£¨¶Ñ¶â»úλÖã©¡¢binUsed£¨»õ¹ñʹÓÃ×´¿ö£©¡¢itemTaken£¨¶Ñ¶â»úÈ¡³ö»õÎï״̬£©¡¢itemStored£¨¶Ñ¶â»ú´æ·Å»õÎï״̬£©¡¢itemInbound£¨¶Ñ¶â»úÈë»õ״̬£©¡¢itemOutbound£¨¶Ñ¶â»ú³ö»õ״̬£©¡¢num£ßInboundItemAt£¨¶Ñ¶â»úÉÏÈë»õ»õÎïµÄÊýÁ¿£©¡¢num£ßOutboundItemAt£¨¶Ñ¶â»úÉϳö»õ»õÎïµÄÊýÁ¿£©¡¢num£ßOutboundItem£¨×ܵijö»õ¶©µ¥ÊýÁ¿£©ºÍnum£ßBinUsed£¨±»Ê¹ÓõĻõ¹ñµÄÊýÁ¿£©¡£

action-fluentÌåÏÖÁìÓòÖнÓÄɵÄÐж¯£¬»®·ÖΪforward£¨ÍùÇ°Òƶ¯£©¡¢back£¨ÍùºóÒƶ¯£©¡¢move£¨Òƶ¯£©¡¢fetch£¨È¡»õ£©¡¢store£¨´æ»õ£©¡¢inbound£¨Èë»õ£©ºÍoutbound£¨³ö»õ£©¡£

d)cpfs¡£cpfs(conditional probability function£¬Ìõ¼þ¸ÅÂʺ¯Êý£©²¿·ÖÊÇʹÓÃRDDL½¨Ä£µÄÖØÒª²¿·Ö£¬¼´fluent±äÁ¿±ØÐëÖ´ÐеÄת»»¡£ËüÄܹ»·´Ó¦×´Ì¬±äÁ¿ÈçºÎ½øÐÐÑÝ»¯¡£±¾ÎIJִ¢µ÷ÀíÁìÓòÖеÄÿ¸öcpf½ÓÄÉÌõ¼þÓï¾äµÈÓï¾ä½á¹¹½âÊ͸÷¸ö±äÁ¿µÄ±ä¸ïÇé¿ö¡£

µ±¶Ñ¶â»úµÖ´ï³ö»õ¿ÚλÖÃÇÒµ±¶Ñ¶â»úÉÏЯ´øµÄ´ý³ö»õ»õÎïÊýÁ¿Îª0ʱ£¬¶Ñ¶â»úÀֳɳö»õ£¬¾ßÌåµÄcpfs½ç˵Ϊ£º

yd2333Ôƶ¥µç×ÓÓÎÏ·ÓÎÏ·(Öйú)ÓÐÏÞ¹«Ë¾



e)reward¡£¸Ã²¿·Ö½ç˵ȷ¶¨ÐԻر¨º¯Êý£¬ÊÜ״̬±äÁ¿ºÍÐж¯±äÁ¿µÄÓ°Ïì¡£±¾²Ö´¢ÁìÓòµÄrewardº¯ÊýÉè¼ÆΪËùÓжѶâ»úÍê³ÉÈë»õºÍ³ö»õÈÎÎñʱµÄʱ¼äÖ®ºÍ£¬¾ßÌåΪ

yd2333Ôƶ¥µç×ÓÓÎÏ·ÓÎÏ·(Öйú)ÓÐÏÞ¹«Ë¾



f)constraints²¿·Ö¡£¸Ã²¿·ÖÖ¸¶¨×´Ì¬Ô¼Êø£¬Ö¼ÔÚÁìÓòʵÀýÔËÐÐ֮ǰ¼ìÑéÄ£ÄâÆÚ¼äÿ¸ö״̬µÄÔ¼ÊøÌõ¼þ£¬ÈçÏÂÔ¼Êøº¯ÊýÌåÏÖ¹ØÓÚÿһ̨¶Ñ¶â»úÀ´Ëµ£¬Ã¿Ò»²½Ö»ÄܽÓÄÉÒ»¸öÐж¯¡£

yd2333Ôƶ¥µç×ÓÓÎÏ·ÓÎÏ·(Öйú)ÓÐÏÞ¹«Ë¾



¾ßÌåµÄ»õ¹ñÊýÁ¿¡¢¶Ñ¶â»úÊýÁ¿µÈÓ°ÏìÕû¸ö²Ö´¢µ÷ÀíÒòËصijõʼ»¯Êý¾Ý¡¢Ä£ÐÍÇó½âÔËÐдÎÊýºÍ²¢ÐÐÔËÐеĶѶâ»úÐж¯ÊýÓÉÎÊÌâʵÀý¸ø³ö¡£Æ¾¾Ý¸ø³öµÄ²Ö´¢¿âλÐÅÏ¢£¬±¾ÎÄÄ£ÐÍ¿ÉÒÔËæʱ¼ÓÈë×îеĿÉÖª±äÁ¿£¨¼´Ó°Ïìµ÷ÀíÀú³ÌµÄÒòËØ£©ºÍÐ޸ıäÁ¿£¨¼´Ó°Ïìµ÷ÀíÀú³ÌµÄÒòËØ£©²ÎÊý£¬²»ÐèÒª±àÒëÖ±½Ó¾ÍÄÜÔËÐС£

4 ʵÑé

a£©ÊµÑéÇé¿ö¡£Windows 10 64λ+CPU 1.70 GHz(i5-4210U£©+ÄÚ´æ8.00 GB£¬Ä£ÐÍhorizon=400¡£ÊµÑéƽ̨ÊÇÔËÓùú¼Ê¸ÅÂʼƻ®´óÈü£¨IPPC£©ÌṩµÄrddlsim(2014£©¼Æ»®Æ÷£¬ËüÄܹ»¶ÔRDDLÃèÊöµÄÁìÓò½øÐÐÍêÈ«½âÎöºÍÅäÖùÀÖµº¯Êý²¢ÊµÏÖÇó½âÕ½ÂÔ£¨Ê½£¨1£©£©µÄÅÌËãÀú³Ì¡£

b£©ÊµÑéÄ£ÐÍ¡£Í¼2Ϊ¸Ã²Ö´¢µ÷ÀíÁìÓòÄ£Ð͵ÄRDDLÓïÒåÄ£ÐÍͼ¡£¸ÃͼΪ¶¯Ì¬±´Ò¶Ë¹ÍøÂ磨dynamic Bayesian network,DBN£©À©Õ¹µÄÓ°Ïìͼ£¨influence diagrams,ID£©£¬ÃèÊöÁ˸òִ¢µ÷ÀíÁìÓòÓ°ÏìÒòËØÖ®¼äµÄ¶ÔÓ¦¹Øϵ¡£ÊµÑé¼ÙÉè»õ²ÖÄ£ÐÍ£¨15¡Á5¡Á2£©ÓÐÈýÌõͨµÀ£¬Á½²ã»õ¹ñ¹²ÓÐ72¸ö£¬Èý̨¶Ñ¶â»ú´ÓÈë»õ¿ÚÈë»õÈ»ºó»®·Ö½øÈë¸÷×ÔµÄͨµÀÒÀ¾Ý¸÷×ԵĻõÎﶩµ¥Òƶ¯µ½¶ÔÓ¦µÄ»õ¹ñλÖýøÐдæ»õ»òÈ¡»õ£¬×îºóÔÚ³ö»õ¿Ú³ö»õ¡£ÊµÑ齫¹¹½¨Íê³ÉµÄÁìÓòÎļþÊäÈë½øÐнâÎö£¬½ÓÄÉrddlsimÌṩµÄËæ»ú²¼¶ûÕ½ÂÔ½øÐÐÇó½â£¬ÁìÓòÎÊÌâÓɳõʼ״̬¾­¹ýһϵÁÐÑÝ»¯×îÖÕµÖ´ïÄ¿±ê״̬£¬×Ô¶¯ÊµÏÖÄ£ÄâЧ¹û¡£

ͼ2 ²Ö´¢µ÷ÀíÁìÓòµÄÓïÒåÄ£ÐÍ

ͼ2 ²Ö´¢µ÷ÀíÁìÓòµÄÓïÒåÄ£ÐÍ  


ʵÑéÇó½â½á¹ûÈçͼ3Ëùʾ£¬¿ÉÒÔ¿´³öÈý̨¶Ñ¶â»ú´ÓÈë»õ¿Ú¿ªÊ¼¸÷×Ô½ÓÄɲî±ðµÄÐж¯Íê³ÉÈë»õ¼°³ö»õÈÎÎñ¡£ÓÉÓÚÔÚrddlsimÖнÓÄɵÄÊÇËæ»ú²¼¶ûÕ½ÂÔ£¬¶Ñ¶â»úµÄÐж¯ÊÇËæ»úµÄ£¬ËùÒÔ´Ó³õʼ״̬µ½Ä¿±êÍê³ÉÈÎÎñ״̬ÿ̨¶Ñ¶â»úËùÓõÄÐж¯ÐòÁÐʵʱ¼äÊDzî±ðµÄ¡£ÁìÓòrewardÖµ½ç˵ΪËùÓжѶâ»úÍê³ÉÈÎÎñµÄʱ¼äÖ®ºÍ£¬ËùÒÔ´Ë´ÎÄ£ÄâʵÑéµÄrewardΪ79¡£Õû¸öÄ£ÄâÀú³Ì²»ÐèÒªÈ˹¤µÄ¸ÉÔ¤£¬Í¨¹ýÐж¯ÐòÁÐͼ¿ÉÒÔÖ±¹ÛµØÁ˽âÕû¸öÄ£ÄâÀú³Ì£¬½âÊÍÐÔÇ¿£¬²¢ÇÒÄܹ»Í¨¹ý²»¾øµØÄ£ÄâÇó½â£¬»ñµÃ½ÏµÍµÄrewardÖµµÄÐж¯Õ½ÂÔ¡£

rddlsim¾«È·ÒªÁìÇó½â´ÎÊýÔ½¶à£¬»ñµÃ×îÓŽâ¿ÉÄÜÐÔÔ½´ó¡£µ«Ëæ×ÅÎÊÌâ¹æÄ£µÄÀ©´ó£¬¹ØÓÚÖеȹæÄ£»òÕß´ó¹æÄ£µÄÎÊÌ⣬ÔÚÓÐÏÞµÄʱ¼äÄÚ²»¿ÉÄÜÇóµÃ×îÓŽâ¡£¹ØÓÚ´ó¹æÄ£µÄÎÊÌ⣬²»ÐèÒªÇóµÃ×îӎ⣬ֻÐèÔÚ¶Ìʱ¼äÄÚÇóµÃ´ÎÓŽâ»òÕßÂúÒâ½â¡£±í1Ϊ¹ØÓÚ²î±ð¹æÄ£µÄÄ£ÐÍÎÊÌ⣬¼Æ»®Æ÷Çó½âµÄƽ¾ùʱ¼ä¡£¼ÙÉèÒ»ÌõͨµÀ¼ÓÉÏÁ½²àµÄ»õ¹ñΪһ¸öµ¥Î»£¬´ÓͼÖпÉÒÔ¿´³öÎÊÌâ¹æÄ£Ô½´ó£¬Çó½âʱ¼äÔ½³¤£¬ÇÒµ±ÎÊÌâ¹æÄ£¹ý´óʱ£¨ÈçͨµÀÊýÄ¿µÖ´ï5ʱ£©Çó½âʱ¼ä¹ý³¤ÉõÖÁÎÞ·¨Çó½â¡£

ͼ3 rddlsim¼Æ»®Æ÷Çó½âÐж¯ÐòÁÐ

ͼ3 rddlsim¼Æ»®Æ÷Çó½âÐж¯ÐòÁР 


  

±í1 ²î±ð¹æÄ£ÎÊÌâÇó½âʱ¼ä



±í1 ²î±ð¹æÄ£ÎÊÌâÇó½âʱ¼ä

5 ½áÊøÓï

±¾ÎÄÖ÷ÒªÊÇ»ùÓÚ¸ÅÂʲ¢Ðмƻ®ÒªÁì¶Ô×Ô¶¯ÎïÁ÷²Ö´¢µ÷ÀíÁìÓò½øÐн¨Ä££¬È»ºóÀûÓüƻ®Æ÷rddlsim¼°Ëæ»ú²¼¶ûÕ½ÂÔ½øÐÐÇó½â¡£±¾ÎĹ¹½¨µÄÁìÓòÄ£ÐÍ°üÀ¨Á˶ԶàÖÖÓ°ÏìÒòËØ¡¢Ô¼ÊøÌõ¼þÒÔ¼°¿ÉÄÜʼþµÄÐÎʽ»¯ÃèÊö£¬¿É½âÊÍÐÔÇ¿£¬Ö±¹ÛÒ׶®£¬¾ßÓнϺõÄÄ£ÄâÐÔÄÜ¡£ÊµÑé½á¹û±êÃ÷£¬±¾ÎÄÒªÁìÄܹ»ÊµÏֶѶâ»úÔÚ»õ²ÖÖÐÈë»õºÍ³ö»õÀú³ÌµÄ×Ô¶¯Ä£Ä⣬²¢¿ÉÒÔƾ¾ÝÄ£Ð͵ÄrewardÖµ½ç˵ʵÏÖ¶ÔÄ£Ð͵ÄÓÅ»¯¡£ËäÈ»£¬¸ÃÁìÓòÄ£ÐÍ»¹±£´æÐí¶àÖµµÃÑо¿ºÍ¸ïеķ½Ãæ¡£ÏÂÒ»²½ÊÂÇéÒª¿¼ÂǸü¶àµÄÔ¼ÊøÌõ¼þÓë¿ÉÄÜʼþ£¬°üÀ¨¶ą̀¶Ñ¶â»úµÄÓµÈû¿ØÖƼ°¶àÖÖ²»È·¶¨Ê¹ʵÈ¡£ÁíÍ⣬»¹ÐèÒª²»¾øµØÓÅ»¯Ä£ÐͲÎÊý£¬µ÷½â¼Æ»®Æ÷Çó½âÕ½ÂÔºÍÌá¸ßÇó½âЧÂÊ¡£


¡¾±¾ÎıêÇ©¡¿

¡¾ÔðÈα༭¡¿yd2333Ôƶ¥µç×ÓÓÎÏ·ÔƲÖ

×îÐÂ×ÊѶ

ÍøÕ¾µØͼ
ÓÑÇéÁ´½Ó£ºÔƶ¥¼¯ÍŹÙÍø  AG¾ÅÓλᠠ918²©ÌìÌÃÏÂÔØ  6686ÌåÓý¹ÙÍø  BevictorΤµÂ  ¿­·¢k8¹ÙÍø  ¾ÅÓλáJ9  ÍòÏó³Ç¹ÙÍø  BevictorΤµÂ  ¿­·¢K8Æì½¢Ìü  Ç§ÒÚÇòÓѻᠠK8¿­·¢Ò»´¥¼´·¢  ½ðÄê»á  qyÇòÓѻᠠ¾ÅÖÝ¿áÓÎ