{"id":3193,"date":"2025-08-18T15:10:24","date_gmt":"2025-08-18T07:10:24","guid":{"rendered":"https:\/\/www.leexinghai.com\/aic\/?p=3193"},"modified":"2025-08-18T15:10:24","modified_gmt":"2025-08-18T07:10:24","slug":"3-2%e6%84%9f%e7%9f%a5%e5%99%a8%e7%ae%97%e6%b3%95","status":"publish","type":"post","link":"https:\/\/www.leexinghai.com\/aic\/3-2%e6%84%9f%e7%9f%a5%e5%99%a8%e7%ae%97%e6%b3%95\/","title":{"rendered":"3.2\u611f\u77e5\u5668\u7b97\u6cd5"},"content":{"rendered":"\n

\u627e\u51fa\u4e00\u4e2a\u5411\u91cf\u03c9\u548c\u4e00\u4e2a\u5e38\u6570b\uff0c\u4f7fi=1......N.<\/p>\n\n\n\n

\uff081\uff09\u82e5y_{i<\/sub>=1\uff0c\u5219\u03c9^{T<\/sup>X_{i<\/sub>+b\uff1e0<\/p>\n\n\n\n}}}

\uff082\uff09\u82e5y_{i<\/sub>=-1\uff0c\u5219\u03c9^{T<\/sup>X_{i<\/sub>+b\uff1c0<\/p>\n\n\n\n}}}

\u611f\u77e5\u5668\u6811\u7b97\u6cd5\u5bfb\u627e\u03c9\uff0cb\u7684\u65b9\u6cd5\uff1a<\/p>\n\n\n\n

\uff081\uff09\u968f\u673a\u9009\u62e9\u03c9\uff0cb<\/p>\n\n\n\n

\uff082\uff09\u53d6\u4e00\u4e2a\u8bad\u7ec3\u6837\u672c\uff08X,Y\uff09<\/p>\n\n\n\n

\u82e5\u03c9^{T<\/sup>X_{i<\/sub>+b\uff1e0\uff0c\u4e14y=-1\uff0c\u6709\u03c9=\u03c9-X\uff0cb=b-1<\/li>\n\n\n\n}}

\u82e5\u03c9^{T<\/sup>X_{i<\/sub>+b\uff1c0\uff0c\u4e14y=1\uff0c\u6709\u03c9=\u03c9+X\uff0cb=b+1<\/li>\n<\/ul>\n\n\n\n\uff083\uff09\u518d\u53d6\u53e6\u4e00\u4e2a\uff08X,Y\uff09\uff0c\u8fd4\u56de\uff082\uff09<\/p>\n\n\n\n
\uff084\uff09\u7ec8\u6b62\u6761\u4ef6<\/mark><\/strong>\uff1a\u76f4\u5230\u6240\u6709\u8f93\u5165\u8f93\u51fa\u5bf9\uff08X,Y\uff09\u90fd\u4e0d\u6ee1\u8db3\uff082\uff09\u4e2d\u7684\u4efb\u4e00\u6761\u4ef6\u4e4b\u4e00\uff0c\u9000\u51fa\u5faa\u73af\u3002<\/p>\n\n\n\n
\u26a0\u53ea\u8981\u8bad\u7ec3\u6570\u636e\u7ebf\u6027\u53ef\u5206\uff0c\u611f\u77e5\u5668\u7b97\u6cd5\u5c31\u4e00\u5b9a\u53ef\u4ee5\u505c\u4e0b\u6765\u3002<\/h2>\n\n\n\n<\/figure>\n\n\n\n\u611f\u77e5\u5668\u7b97\u6cd5\u7684\u6536\u655b\u5b9a\u7406<\/p>\n\n\n\n
<\/figure>\n\n\n\n\u601d\u8003\uff1a\u73b0\u5728\u6709\u8fd9\u4e2a\u95ee\u9898\uff1a\u5982\u679c\u8bad\u7ec3\u6570\u636e\u96c6\u662f\u7ebf\u6027\u4e0d\u53ef\u5206\u7684\uff0c\u90a3\u4e48\u611f\u77e5\u5668\u7b97\u6cd5\u662f\u5426\u8fd8\u4f1a\u6536\u655b\uff1f\u5982\u679c\u6b64\u65f6\u611f\u77e5\u5668\u7b97\u6cd5\u4e0d\u6536\u655b\uff0c\u80fd\u5426\u6539\u9020\u8fd9\u4e2a\u7b97\u6cd5\uff0c\u4f7f\u5b83\u5728\u7ebf\u6027\u4e0d\u53ef\u5206\u7684\u60c5\u51b5\u4e0b\u81ea\u52a8\u7ec8\u6b62\u5faa\u73af\uff1f<\/h2>\n\n\n\n\u7ed3\u8bba\u5148\u884c<\/strong><\/p>\n\n\n\n
\n\u6807\u51c6\u611f\u77e5\u5668<\/strong>\u4ec5\u5728\u6570\u636e \u7ebf\u6027\u53ef\u5206<\/strong> \u65f6\u4fdd\u8bc1\u6536\u655b\uff1b<\/li>\n\n\n\n
\u5f53\u6570\u636e \u7ebf\u6027\u4e0d\u53ef\u5206<\/strong> \u65f6\uff0c\u5b83\u4f1a\u201c\u6c38\u8fdc\u201d\u66f4\u65b0\u4e0b\u53bb\uff08\u7406\u8bba\u4e0a\u66f4\u65b0\u6b21\u6570\u65e0\u4e0a\u9650\uff0c\u5b9e\u8df5\u4e2d\u8868\u73b0\u4e3a\u5faa\u73af\u6216\u6743\u5411\u91cf\u7206\u957f\uff09\uff1b<\/li>\n\n\n\n
\u53ef\u4ee5\u901a\u8fc7 Pocket \u611f\u77e5\u5668\u3001\u5e73\u5747\u611f\u77e5\u5668\u3001\u9650\u6b65\u65e9\u505c\u7b49\u6539\u9020<\/strong>\uff0c\u8ba9\u7b97\u6cd5\u5728\u4e0d\u53ef\u5206\u573a\u666f\u4e0b \u81ea\u52a8\u7ec8\u6b62<\/strong>\uff0c\u540c\u65f6\u7ed9\u51fa\u4e00\u4e2a\u201c\u5c3d\u53ef\u80fd\u597d\u201d\u7684\u6743\u5411\u91cf\u3002<\/li>\n<\/ul>\n\n\n\n
\n\n\n\n1 \u4e3a\u4ec0\u4e48\u6807\u51c6\u611f\u77e5\u5668\u4e0d\u6536\u655b\uff1f<\/h2>\n\n\n\n\n\u66f4\u65b0\u89c4\u5219 \u03c9\u2190\u03c9+\u03b7y_{i<\/sub>x_{i<\/sub>,\u2005\u200ab\u2190b+\u03b7y_{i<\/sub>\u53ea\u8981\u8fd8\u5b58\u5728\u8bef\u5206\u7c7b\u6837\u672c\u5c31\u6301\u7eed\u8fdb\u884c\u3002<\/li>\n\n\n\n}}}
\u7ebf\u6027\u4e0d\u53ef\u5206 \u21d2 \u4efb\u610f \u03c9,b\u90fd\u6709\u8bef\u5206\u6837\u672c \u21d2 \u6761\u4ef6\u6c38\u8fdc\u6ee1\u8db3 \u21d2 \u5faa\u73af<\/strong>\u3002<\/li>\n\n\n\n
\u5178\u578b\u73b0\u8c61\uff1a\u6743\u5411\u91cf\u8303\u6570\u968f\u8fed\u4ee3\u5448\u7ebf\u6027\u6216\u5e73\u65b9\u6839\u589e\u957f\uff0c\u9519\u8bef\u6570\u5728\u4e00\u4e2a\u533a\u95f4\u5185\u6765\u56de\u632f\u8361\u3002<\/li>\n<\/ul>\n\n\n\n
\n\n\n\n2 \u4e09\u79cd\u5e38\u7528\u201c\u81ea\u52a8\u505c\u6b62\u201d\u6539\u9020\u65b9\u6848<\/h2>\n\n\n\n\u6539\u9020<\/th> \u601d\u60f3<\/th> \u81ea\u52a8\u7ec8\u6b62\u6761\u4ef6<\/th> \u6536\u655b\u5230\u4ec0\u4e48<\/th><\/tr><\/thead>
Pocket \u611f\u77e5\u5668<\/strong><\/td> \u7ef4\u62a4\u4e00\u4efd\u201c\u53e3\u888b\u201d\u6743\u5411\u91cf \u03c9*\u53ca\u5176\u9519\u8bef\u7387 E*\u3002\u9047\u5230\u66f4\u4f18\u89e3\u5c31\u66ff\u6362\u53e3\u888b\u3002<\/td> \u9884\u8bbe \u8fed\u4ee3\u4e0a\u9650<\/strong> \u6216 \u8fde\u7eed\u82e5\u5e72\u8f6e\u65e0\u6539\u8fdb<\/strong><\/td> \u8fd4\u56de\u5386\u53f2\u6700\u4f18 \u03c9*\uff1b\u5373\u4f7f\u4e0d\u53ef\u5206\u4e5f\u7a33\u5b9a<\/td><\/tr>
\u5e73\u5747\u611f\u77e5\u5668<\/strong><\/td> \u5c06\u5386\u6b21\u6743\u5411\u91cf\u6c42\u5e73\u5747 \u03c9\u02c9=1\/T*\u2211[T t=1]\u03c9_{t<\/sub><\/td>} \u9884\u8bbe\u6700\u5927 epoch \u6570<\/td> \u7406\u8bba\u4e0a\u7b49\u4ef7\u4e8e\u5bf9 hinge \u635f\u5931\u505a SGD\uff0c\u6cdb\u5316\u8bef\u5dee\u66f4\u4f4e<\/td><\/tr>
\u9650\u6b65 + \u65e9\u505c<\/strong><\/td> \u56fa\u5b9a\u6700\u5927\u8fed\u4ee3 T_{max\u2061<\/sub>\u200b\uff0c\u6bcf\u8f6e\u7edf\u8ba1\u8bef\u5206\u7c7b\u6570\uff1b\u82e5\u5728\u9a8c\u8bc1\u96c6\u4e0a N \u8f6e\u4e0d\u4e0b\u964d<\/strong> \u5219\u505c\u6b62<\/td>} \u8bef\u5dee\u201c\u505c\u6ede\u201d<\/td> \u8fd4\u56de\u6700\u8fd1\u4e00\u6b21\u6700\u4f73 \u03c9<\/td><\/tr><\/tbody><\/table><\/figure>\n\n\n\n\nPocket \u4e0e\u5e73\u5747\u611f\u77e5\u5668\u90fd\u662f\u201c\u611f\u77e5\u5668\u65cf\u201d\u4e2d\u6700\u7ecf\u5178\u3001\u6548\u679c\u4e5f\u6700\u7a33\u7684\u4e24\u62db\u3002<\/p>\n<\/blockquote>\n\n\n\n
\n\n\n\n2.1 Pocket \u611f\u77e5\u5668\u6838\u5fc3\u4ee3\u7801\u7247\u6bb5<\/h3>\n\n\n\ndef pocket_perceptron(X, y, T_max=10000):\n w = np.zeros(X.shape[1]); b = 0\n w_best, b_best = w.copy(), b\n err_best = (y*np.sign(X@w + b) <= 0).mean()\n\n for t in range(T_max):\n for i in range(len(y)):\n if y[i]*(X[i]@w + b) <= 0:\n w += y[i]*X[i]\n b += y[i]\n err_cur = (y*np.sign(X@w + b) <= 0).mean()\n if err_cur < err_best:\n w_best, b_best = w.copy(), b\n err_best = err_cur\n # \u8fde\u7eed 10 \u8f6e\u6ca1\u6709\u6539\u8fdb\u5c31\u63d0\u524d\u505c\n if patience_counter(err_best, t): break\n return w_best, b_best\n<\/code><\/pre>\n\n\n\n
\n\n\n\n2.2 \u5e73\u5747\u611f\u77e5\u5668\uff08Averaged Perceptron\uff09<\/h3>\n\n\n\n\u8bad\u7ec3\u65f6\u7d2f\u79ef\u6743\u5411\u91cf\uff1a<\/p>\n\n\n\n
<\/figure>\n\n\n\n\u7406\u8bba\uff1a\u7b49\u4ef7\u4e8e\u5728\u6240\u6709\u5386\u53f2\u8d85\u5e73\u9762\u4e0a\u505a\u6295\u7968\uff0c\u53ef\u663e\u8457\u51cf\u5c0f\u65b9\u5dee\uff1b\u5728\u4e0d\u53ef\u5206\u6570\u636e\u4e0a\u5b9e\u9a8c\u6548\u679c\u901a\u5e38\u4f18\u4e8e\u5355\u4e00 \u03c9_{t<\/sub>\u200b\u3002<\/p>\n\n\n\n}
\u7ec8\u6b62\uff1a\u56fa\u5b9a epoch \u6570\u6216\u9a8c\u8bc1\u96c6\u65e9\u505c\u3002<\/p>\n\n\n\n
\n\n\n\n3 \u5176\u4ed6\u53ef\u9009\u7b56\u7565<\/h2>\n\n\n\n\n\u52a0\u5165\u6b63\u5219 \/ \u635f\u5931\u51fd\u6570\u66ff\u6362<\/strong>\n\n\u6362\u6210 \u5e73\u65b9hinge \u635f\u5931<\/strong>\uff0bSGD \u21d2 \u5b9e\u9645\u5c31\u662f \u7ebf\u6027 SVM<\/strong>\u3002<\/li>\n\n\n\n
\u6362\u6210 logistic \u635f\u5931<\/strong> \u21d2 \u9000\u5316\u4e3a \u903b\u8f91\u56de\u5f52<\/strong>\u3002
\u8fd9\u4e24\u79cd\u90fd\u5728\u4e0d\u53ef\u5206\u6570\u636e\u4e0a\u5929\u7136\u6709\u6700\u4f18\u89e3\uff0c\u4e14\u4e00\u6b21\u4f18\u5316\u5373\u53ef\u7ed3\u675f\u3002<\/li>\n<\/ul>\n<\/li>\n\n\n\n
\u6838\u6280\u5de7<\/strong>\n\n\u7528\u6838\u51fd\u6570\u628a\u6570\u636e\u6620\u5c04\u5230\u66f4\u9ad8\u7ef4\uff0c\u82e5\u6620\u5c04\u540e\u53ef\u5206\uff0c\u611f\u77e5\u5668\u53c8\u53ef\u6536\u655b\uff1b<\/li>\n\n\n\n
\u4f46\u540c\u6837\u9700 Pocket\/\u5e73\u5747\u65b9\u6848\u6765\u5904\u7406\u4ecd\u4e0d\u53ef\u5206\u7684\u60c5\u51b5\u3002<\/li>\n<\/ul>\n<\/li>\n\n\n\n
\u8bbe\u5b9a\u5b66\u4e60\u7387\u8870\u51cf<\/strong>\n\n\u4ee4 \u03b7_{t<\/sub>=\u03b7_{0<\/sub>\/(1+t)\u3002\u5373\u4f7f\u4e0d\u6536\u655b\uff0c\u66f4\u65b0\u5e45\u5ea6\u8d8b\u96f6\uff0c\u8fed\u4ee3\u5e8f\u5217 {\u03c9_{t<\/sub>}\u6709\u754c \u2192 \u7406\u8bba\u4e0a\u53ef\u8bc1\u660e\u6743\u5411\u91cf\u6536\u655b\u5230\u4e00\u7a33\u6001\u503c\u3002<\/li>\n\n\n\n}}}
\u5b9e\u8df5\u4e2d\u5e38\u914d\u5408\u65e9\u505c\u4e00\u8d77\u4f7f\u7528\u3002<\/li>\n<\/ul>\n<\/li>\n<\/ol>\n\n\n\n
\n\n\n\n4 \u5c0f\u7ed3<\/h2>\n\n\n\n\n\u7ebf\u6027\u4e0d\u53ef\u5206 \u21d2 \u6807\u51c6\u611f\u77e5\u5668\u4e0d\u6536\u655b<\/strong>\u3002<\/li>\n\n\n\n
Pocket\u3001\u5e73\u5747\u611f\u77e5\u5668\u3001\u9650\u6b65\u65e9\u505c<\/strong> \u7b49\u6539\u9020\uff0c\u53ef\u5728\u7ed9\u5b9a\u6761\u4ef6\u4e0b\u81ea\u52a8\u505c\u6b62\uff0c\u5e76\u8fd4\u56de\u6700\u4f18\u6216\u5e73\u5747\u6743\u5411\u91cf\u3002<\/li>\n\n\n\n
\u82e5\u60f3\u8981\u201c\u4ece\u6839\u672c\u4e0a\u201d\u5f97\u5230\u51f8\u4f18\u5316\u89e3\uff0c\u53ef\u76f4\u63a5\u6362\u6210 SVM \/ Logistic<\/strong> \u7b49\u6709\u635f\u5931\u6700\u5c0f\u5316\u76ee\u6807\u7684\u6a21\u578b\u3002<\/li>\n<\/ul>\n\n\n\n\u8fd9\u6837\u5c31\u80fd\u5728\u4efb\u4f55\u6570\u636e\u5206\u5e03\u4e0b\u8ba9\u201c\u611f\u77e5\u5668\u601d\u60f3\u201d\u5b89\u5168\u843d\u5730\uff0c\u800c\u4e0d\u4f1a\u9677\u5165\u65e0\u9650\u5faa\u73af\u3002<\/p>\n","protected":false},"excerpt":{"rendered":"
\u627e\u51fa\u4e00\u4e2a\u5411\u91cf\u03c9\u548c\u4e00\u4e2a\u5e38\u6570b\uff0c\u4f7fi=1……N. \uff081\uff09\u82e5yi=1\uff0c\u5219\u03c9TXi+b\uff1e0 \uff082\uff09\u82e5yi=-1 […]<\/p>\n","protected":false},"author":1,"featured_media":3045,"comment_status":"closed","ping_status":"closed","sticky":false,"template":"","format":"standard","meta":{"footnotes":""},"categories":[63],"tags":[],"class_list":["post-3193","post","type-post","status-publish","format-standard","has-post-thumbnail","hentry","category-mlsh"],"_links":{"self":[{"href":"https:\/\/www.leexinghai.com\/aic\/wp-json\/wp\/v2\/posts\/3193","targetHints":{"allow":["GET"]}}],"collection":[{"href":"https:\/\/www.leexinghai.com\/aic\/wp-json\/wp\/v2\/posts"}],"about":[{"href":"https:\/\/www.leexinghai.com\/aic\/wp-json\/wp\/v2\/types\/post"}],"author":[{"embeddable":true,"href":"https:\/\/www.leexinghai.com\/aic\/wp-json\/wp\/v2\/users\/1"}],"replies":[{"embeddable":true,"href":"https:\/\/www.leexinghai.com\/aic\/wp-json\/wp\/v2\/comments?post=3193"}],"version-history":[{"count":1,"href":"https:\/\/www.leexinghai.com\/aic\/wp-json\/wp\/v2\/posts\/3193\/revisions"}],"predecessor-version":[{"id":3197,"href":"https:\/\/www.leexinghai.com\/aic\/wp-json\/wp\/v2\/posts\/3193\/revisions\/3197"}],"wp:featuredmedia":[{"embeddable":true,"href":"https:\/\/www.leexinghai.com\/aic\/wp-json\/wp\/v2\/media\/3045"}],"wp:attachment":[{"href":"https:\/\/www.leexinghai.com\/aic\/wp-json\/wp\/v2\/media?parent=3193"}],"wp:term":[{"taxonomy":"category","embeddable":true,"href":"https:\/\/www.leexinghai.com\/aic\/wp-json\/wp\/v2\/categories?post=3193"},{"taxonomy":"post_tag","embeddable":true,"href":"https:\/\/www.leexinghai.com\/aic\/wp-json\/wp\/v2\/tags?post=3193"}],"curies":[{"name":"wp","href":"https:\/\/api.w.org\/{rel}","templated":true}]}}}}