Skip to content

D
deep_learning_CNN
提交 fb649006 作者: 龙婷
操作

update

上级 2d5207f5
正在显示 包含 4968 行增加 和 0 行删除
前馈神经网络shr/前馈神经网络（多层感知机）.md 0 → 100644
# 前馈神经网络（多层感知机）
# 前馈神经网络（多层感知机）
课程ppt-带讲课备注
课程书籍：神经网络与深度学习（第四章）/深度学习花书（第六章）
课程参考视频：https://www.bilibili.com/video/BV13b4y1177W?p=29&vd_source=149ba9b3f35d06876e4f13eb30ed541e
实例：单隐藏层实现前馈多层感知机用来对Fashion-MNIST图像数据集进行分类
实例代码参考教学视频：https://www.bilibili.com/video/BV1hh411U7gn/?p=3&share_source=copy_web&vd_source=50692d688419fbaefda2a76f9eb6adbe
课程时间：2023年10月31日
主讲人：孙浩然
前馈神经网络shr/实例代码/image-classification-dataset.ipynb 0 → 100644
This source diff could not be displayed because it is too large. You can view the blob instead.
前馈神经网络shr/实例代码/mlp-concise.ipynb 0 → 100644
{
{
"cells": [
{
"cell_type": "markdown",
"id": "d5217b24",
"metadata": {
"origin_pos": 0
},
"source": [
"# 多层感知机的简洁实现\n",
":label:`sec_mlp_concise`\n",
"\n",
"本节将介绍(**通过高级API更简洁地实现多层感知机**)。\n"
]
},
{
"cell_type": "code",
"execution_count": 1,
"id": "f4b9d183",
"metadata": {
"execution": {
"iopub.execute_input": "2023-08-18T07:04:20.711610Z",
"iopub.status.busy": "2023-08-18T07:04:20.711337Z",
"iopub.status.idle": "2023-08-18T07:04:22.715766Z",
"shell.execute_reply": "2023-08-18T07:04:22.714884Z"
},
"origin_pos": 2,
"tab": [
"pytorch"
]
},
"outputs": [],
"source": [
"import torch\n",
"from torch import nn\n",
"from d2l import torch as d2l"
]
},
{
"cell_type": "markdown",
"id": "d1b8af0c",
"metadata": {
"origin_pos": 5
},
"source": [
"## 模型\n",
"\n",
"与softmax回归的简洁实现（ :numref:`sec_softmax_concise`）相比，\n",
"唯一的区别是我们添加了2个全连接层（之前我们只添加了1个全连接层）。\n",
"第一层是[**隐藏层**]，它(**包含256个隐藏单元，并使用了ReLU激活函数**)。\n",
"第二层是输出层。\n"
]
},
{
"cell_type": "code",
"execution_count": 2,
"id": "a11cfbe9",
"metadata": {
"execution": {
"iopub.execute_input": "2023-08-18T07:04:22.719981Z",
"iopub.status.busy": "2023-08-18T07:04:22.719298Z",
"iopub.status.idle": "2023-08-18T07:04:22.748628Z",
"shell.execute_reply": "2023-08-18T07:04:22.747813Z"
},
"origin_pos": 7,
"tab": [
"pytorch"
]
},
"outputs": [],
"source": [
"net = nn.Sequential(nn.Flatten(),\n",
" nn.Linear(784, 256),\n",
" nn.ReLU(),\n",
" nn.Linear(256, 10))\n",
"\n",
"def init_weights(m):\n",
" if type(m) == nn.Linear:\n",
" nn.init.normal_(m.weight, std=0.01)\n",
"\n",
"net.apply(init_weights);"
]
},
{
"cell_type": "markdown",
"id": "f5aceed6",
"metadata": {
"origin_pos": 10
},
"source": [
"[**训练过程**]的实现与我们实现softmax回归时完全相同，\n",
"这种模块化设计使我们能够将与模型架构有关的内容独立出来。\n"
]
},
{
"cell_type": "code",
"execution_count": 3,
"id": "b23e8ab9",
"metadata": {
"execution": {
"iopub.execute_input": "2023-08-18T07:04:22.753701Z",
"iopub.status.busy": "2023-08-18T07:04:22.753406Z",
"iopub.status.idle": "2023-08-18T07:04:22.758051Z",
"shell.execute_reply": "2023-08-18T07:04:22.757284Z"
},
"origin_pos": 12,
"tab": [
"pytorch"
]
},
"outputs": [],
"source": [
"batch_size, lr, num_epochs = 256, 0.1, 10\n",
"loss = nn.CrossEntropyLoss(reduction='none')\n",
"trainer = torch.optim.SGD(net.parameters(), lr=lr)"
]
},
{
"cell_type": "code",
"execution_count": 4,
"id": "78ac9bf1",
"metadata": {
"execution": {
"iopub.execute_input": "2023-08-18T07:04:22.761842Z",
"iopub.status.busy": "2023-08-18T07:04:22.761295Z",
"iopub.status.idle": "2023-08-18T07:05:05.308680Z",
"shell.execute_reply": "2023-08-18T07:05:05.307786Z"
},
"origin_pos": 15,
"tab": [
"pytorch"
]
},
"outputs": [
{
"data": {
"image/svg+xml": [
"<?xml version=\"1.0\" encoding=\"utf-8\" standalone=\"no\"?>\n",
"<!DOCTYPE svg PUBLIC \"-//W3C//DTD SVG 1.1//EN\"\n",
" \"http://www.w3.org/Graphics/SVG/1.1/DTD/svg11.dtd\">\n",
"<svg xmlns:xlink=\"http://www.w3.org/1999/xlink\" width=\"238.965625pt\" height=\"180.65625pt\" viewBox=\"0 0 238.965625 180.65625\" xmlns=\"http://www.w3.org/2000/svg\" version=\"1.1\">\n",
" <metadata>\n",
" <rdf:RDF xmlns:dc=\"http://purl.org/dc/elements/1.1/\" xmlns:cc=\"http://creativecommons.org/ns#\" xmlns:rdf=\"http://www.w3.org/1999/02/22-rdf-syntax-ns#\">\n",
" <cc:Work>\n",
" <dc:type rdf:resource=\"http://purl.org/dc/dcmitype/StillImage\"/>\n",
" <dc:date>2023-08-18T07:05:05.270258</dc:date>\n",
" <dc:format>image/svg+xml</dc:format>\n",
" <dc:creator>\n",
" <cc:Agent>\n",
" <dc:title>Matplotlib v3.5.1, https://matplotlib.org/</dc:title>\n",
" </cc:Agent>\n",
" </dc:creator>\n",
" </cc:Work>\n",
" </rdf:RDF>\n",
" </metadata>\n",
" <defs>\n",
" <style type=\"text/css\">*{stroke-linejoin: round; stroke-linecap: butt}</style>\n",
" </defs>\n",
" <g id=\"figure_1\">\n",
" <g id=\"patch_1\">\n",
" <path d=\"M 0 180.65625 \n",
"L 238.965625 180.65625 \n",
"L 238.965625 0 \n",
"L 0 0 \n",
"L 0 180.65625 \n",
"z\n",
"\" style=\"fill: none\"/>\n",
" </g>\n",
" <g id=\"axes_1\">\n",
" <g id=\"patch_2\">\n",
" <path d=\"M 30.103125 143.1 \n",
"L 225.403125 143.1 \n",
"L 225.403125 7.2 \n",
"L 30.103125 7.2 \n",
"z\n",
"\" style=\"fill: #ffffff\"/>\n",
" </g>\n",
" <g id=\"matplotlib.axis_1\">\n",
" <g id=\"xtick_1\">\n",
" <g id=\"line2d_1\">\n",
" <path d=\"M 51.803125 143.1 \n",
"L 51.803125 7.2 \n",
"\" clip-path=\"url(#p38f7277f50)\" style=\"fill: none; stroke: #b0b0b0; stroke-width: 0.8; stroke-linecap: square\"/>\n",
" </g>\n",
" <g id=\"line2d_2\">\n",
" <defs>\n",
" <path id=\"m69cc5df15a\" d=\"M 0 0 \n",
"L 0 3.5 \n",
"\" style=\"stroke: #000000; stroke-width: 0.8\"/>\n",
" </defs>\n",
" <g>\n",
" <use xlink:href=\"#m69cc5df15a\" x=\"51.803125\" y=\"143.1\" style=\"stroke: #000000; stroke-width: 0.8\"/>\n",
" </g>\n",
" </g>\n",
" <g id=\"text_1\">\n",
" <!-- 2 -->\n",
" <g transform=\"translate(48.621875 157.698438)scale(0.1 -0.1)\">\n",
" <defs>\n",
" <path id=\"DejaVuSans-32\" d=\"M 1228 531 \n",
"L 3431 531 \n",
"L 3431 0 \n",
"L 469 0 \n",
"L 469 531 \n",
"Q 828 903 1448 1529 \n",
"Q 2069 2156 2228 2338 \n",
"Q 2531 2678 2651 2914 \n",
"Q 2772 3150 2772 3378 \n",
"Q 2772 3750 2511 3984 \n",
"Q 2250 4219 1831 4219 \n",
"Q 1534 4219 1204 4116 \n",
"Q 875 4013 500 3803 \n",
"L 500 4441 \n",
"Q 881 4594 1212 4672 \n",
"Q 1544 4750 1819 4750 \n",
"Q 2544 4750 2975 4387 \n",
"Q 3406 4025 3406 3419 \n",
"Q 3406 3131 3298 2873 \n",
"Q 3191 2616 2906 2266 \n",
"Q 2828 2175 2409 1742 \n",
"Q 1991 1309 1228 531 \n",
"z\n",
"\" transform=\"scale(0.015625)\"/>\n",
" </defs>\n",
" <use xlink:href=\"#DejaVuSans-32\"/>\n",
" </g>\n",
" </g>\n",
" </g>\n",
" <g id=\"xtick_2\">\n",
" <g id=\"line2d_3\">\n",
" <path d=\"M 95.203125 143.1 \n",
"L 95.203125 7.2 \n",
"\" clip-path=\"url(#p38f7277f50)\" style=\"fill: none; stroke: #b0b0b0; stroke-width: 0.8; stroke-linecap: square\"/>\n",
" </g>\n",
" <g id=\"line2d_4\">\n",
" <g>\n",
" <use xlink:href=\"#m69cc5df15a\" x=\"95.203125\" y=\"143.1\" style=\"stroke: #000000; stroke-width: 0.8\"/>\n",
" </g>\n",
" </g>\n",
" <g id=\"text_2\">\n",
" <!-- 4 -->\n",
" <g transform=\"translate(92.021875 157.698438)scale(0.1 -0.1)\">\n",
" <defs>\n",
" <path id=\"DejaVuSans-34\" d=\"M 2419 4116 \n",
"L 825 1625 \n",
"L 2419 1625 \n",
"L 2419 4116 \n",
"z\n",
"M 2253 4666 \n",
"L 3047 4666 \n",
"L 3047 1625 \n",
"L 3713 1625 \n",
"L 3713 1100 \n",
"L 3047 1100 \n",
"L 3047 0 \n",
"L 2419 0 \n",
"L 2419 1100 \n",
"L 313 1100 \n",
"L 313 1709 \n",
"L 2253 4666 \n",
"z\n",
"\" transform=\"scale(0.015625)\"/>\n",
" </defs>\n",
" <use xlink:href=\"#DejaVuSans-34\"/>\n",
" </g>\n",
" </g>\n",
" </g>\n",
" <g id=\"xtick_3\">\n",
" <g id=\"line2d_5\">\n",
" <path d=\"M 138.603125 143.1 \n",
"L 138.603125 7.2 \n",
"\" clip-path=\"url(#p38f7277f50)\" style=\"fill: none; stroke: #b0b0b0; stroke-width: 0.8; stroke-linecap: square\"/>\n",
" </g>\n",
" <g id=\"line2d_6\">\n",
" <g>\n",
" <use xlink:href=\"#m69cc5df15a\" x=\"138.603125\" y=\"143.1\" style=\"stroke: #000000; stroke-width: 0.8\"/>\n",
" </g>\n",
" </g>\n",
" <g id=\"text_3\">\n",
" <!-- 6 -->\n",
" <g transform=\"translate(135.421875 157.698438)scale(0.1 -0.1)\">\n",
" <defs>\n",
" <path id=\"DejaVuSans-36\" d=\"M 2113 2584 \n",
"Q 1688 2584 1439 2293 \n",
"Q 1191 2003 1191 1497 \n",
"Q 1191 994 1439 701 \n",
"Q 1688 409 2113 409 \n",
"Q 2538 409 2786 701 \n",
"Q 3034 994 3034 1497 \n",
"Q 3034 2003 2786 2293 \n",
"Q 2538 2584 2113 2584 \n",
"z\n",
"M 3366 4563 \n",
"L 3366 3988 \n",
"Q 3128 4100 2886 4159 \n",
"Q 2644 4219 2406 4219 \n",
"Q 1781 4219 1451 3797 \n",
"Q 1122 3375 1075 2522 \n",
"Q 1259 2794 1537 2939 \n",
"Q 1816 3084 2150 3084 \n",
"Q 2853 3084 3261 2657 \n",
"Q 3669 2231 3669 1497 \n",
"Q 3669 778 3244 343 \n",
"Q 2819 -91 2113 -91 \n",
"Q 1303 -91 875 529 \n",
"Q 447 1150 447 2328 \n",
"Q 447 3434 972 4092 \n",
"Q 1497 4750 2381 4750 \n",
"Q 2619 4750 2861 4703 \n",
"Q 3103 4656 3366 4563 \n",
"z\n",
"\" transform=\"scale(0.015625)\"/>\n",
" </defs>\n",
" <use xlink:href=\"#DejaVuSans-36\"/>\n",
" </g>\n",
" </g>\n",
" </g>\n",
" <g id=\"xtick_4\">\n",
" <g id=\"line2d_7\">\n",
" <path d=\"M 182.003125 143.1 \n",
"L 182.003125 7.2 \n",
"\" clip-path=\"url(#p38f7277f50)\" style=\"fill: none; stroke: #b0b0b0; stroke-width: 0.8; stroke-linecap: square\"/>\n",
" </g>\n",
" <g id=\"line2d_8\">\n",
" <g>\n",
" <use xlink:href=\"#m69cc5df15a\" x=\"182.003125\" y=\"143.1\" style=\"stroke: #000000; stroke-width: 0.8\"/>\n",
" </g>\n",
" </g>\n",
" <g id=\"text_4\">\n",
" <!-- 8 -->\n",
" <g transform=\"translate(178.821875 157.698438)scale(0.1 -0.1)\">\n",
" <defs>\n",
" <path id=\"DejaVuSans-38\" d=\"M 2034 2216 \n",
"Q 1584 2216 1326 1975 \n",
"Q 1069 1734 1069 1313 \n",
"Q 1069 891 1326 650 \n",
"Q 1584 409 2034 409 \n",
"Q 2484 409 2743 651 \n",
"Q 3003 894 3003 1313 \n",
"Q 3003 1734 2745 1975 \n",
"Q 2488 2216 2034 2216 \n",
"z\n",
"M 1403 2484 \n",
"Q 997 2584 770 2862 \n",
"Q 544 3141 544 3541 \n",
"Q 544 4100 942 4425 \n",
"Q 1341 4750 2034 4750 \n",
"Q 2731 4750 3128 4425 \n",
"Q 3525 4100 3525 3541 \n",
"Q 3525 3141 3298 2862 \n",
"Q 3072 2584 2669 2484 \n",
"Q 3125 2378 3379 2068 \n",
"Q 3634 1759 3634 1313 \n",
"Q 3634 634 3220 271 \n",
"Q 2806 -91 2034 -91 \n",
"Q 1263 -91 848 271 \n",
"Q 434 634 434 1313 \n",
"Q 434 1759 690 2068 \n",
"Q 947 2378 1403 2484 \n",
"z\n",
"M 1172 3481 \n",
"Q 1172 3119 1398 2916 \n",
"Q 1625 2713 2034 2713 \n",
"Q 2441 2713 2670 2916 \n",
"Q 2900 3119 2900 3481 \n",
"Q 2900 3844 2670 4047 \n",
"Q 2441 4250 2034 4250 \n",
"Q 1625 4250 1398 4047 \n",
"Q 1172 3844 1172 3481 \n",
"z\n",
"\" transform=\"scale(0.015625)\"/>\n",
" </defs>\n",
" <use xlink:href=\"#DejaVuSans-38\"/>\n",
" </g>\n",
" </g>\n",
" </g>\n",
" <g id=\"xtick_5\">\n",
" <g id=\"line2d_9\">\n",
" <path d=\"M 225.403125 143.1 \n",
"L 225.403125 7.2 \n",
"\" clip-path=\"url(#p38f7277f50)\" style=\"fill: none; stroke: #b0b0b0; stroke-width: 0.8; stroke-linecap: square\"/>\n",
" </g>\n",
" <g id=\"line2d_10\">\n",
" <g>\n",
" <use xlink:href=\"#m69cc5df15a\" x=\"225.403125\" y=\"143.1\" style=\"stroke: #000000; stroke-width: 0.8\"/>\n",
" </g>\n",
" </g>\n",
" <g id=\"text_5\">\n",
" <!-- 10 -->\n",
" <g transform=\"translate(219.040625 157.698438)scale(0.1 -0.1)\">\n",
" <defs>\n",
" <path id=\"DejaVuSans-31\" d=\"M 794 531 \n",
"L 1825 531 \n",
"L 1825 4091 \n",
"L 703 3866 \n",
"L 703 4441 \n",
"L 1819 4666 \n",
"L 2450 4666 \n",
"L 2450 531 \n",
"L 3481 531 \n",
"L 3481 0 \n",
"L 794 0 \n",
"L 794 531 \n",
"z\n",
"\" transform=\"scale(0.015625)\"/>\n",
" <path id=\"DejaVuSans-30\" d=\"M 2034 4250 \n",
"Q 1547 4250 1301 3770 \n",
"Q 1056 3291 1056 2328 \n",
"Q 1056 1369 1301 889 \n",
"Q 1547 409 2034 409 \n",
"Q 2525 409 2770 889 \n",
"Q 3016 1369 3016 2328 \n",
"Q 3016 3291 2770 3770 \n",
"Q 2525 4250 2034 4250 \n",
"z\n",
"M 2034 4750 \n",
"Q 2819 4750 3233 4129 \n",
"Q 3647 3509 3647 2328 \n",
"Q 3647 1150 3233 529 \n",
"Q 2819 -91 2034 -91 \n",
"Q 1250 -91 836 529 \n",
"Q 422 1150 422 2328 \n",
"Q 422 3509 836 4129 \n",
"Q 1250 4750 2034 4750 \n",
"z\n",
"\" transform=\"scale(0.015625)\"/>\n",
" </defs>\n",
" <use xlink:href=\"#DejaVuSans-31\"/>\n",
" <use xlink:href=\"#DejaVuSans-30\" x=\"63.623047\"/>\n",
" </g>\n",
" </g>\n",
" </g>\n",
" <g id=\"text_6\">\n",
" <!-- epoch -->\n",
" <g transform=\"translate(112.525 171.376563)scale(0.1 -0.1)\">\n",
" <defs>\n",
" <path id=\"DejaVuSans-65\" d=\"M 3597 1894 \n",
"L 3597 1613 \n",
"L 953 1613 \n",
"Q 991 1019 1311 708 \n",
"Q 1631 397 2203 397 \n",
"Q 2534 397 2845 478 \n",
"Q 3156 559 3463 722 \n",
"L 3463 178 \n",
"Q 3153 47 2828 -22 \n",
"Q 2503 -91 2169 -91 \n",
"Q 1331 -91 842 396 \n",
"Q 353 884 353 1716 \n",
"Q 353 2575 817 3079 \n",
"Q 1281 3584 2069 3584 \n",
"Q 2775 3584 3186 3129 \n",
"Q 3597 2675 3597 1894 \n",
"z\n",
"M 3022 2063 \n",
"Q 3016 2534 2758 2815 \n",
"Q 2500 3097 2075 3097 \n",
"Q 1594 3097 1305 2825 \n",
"Q 1016 2553 972 2059 \n",
"L 3022 2063 \n",
"z\n",
"\" transform=\"scale(0.015625)\"/>\n",
" <path id=\"DejaVuSans-70\" d=\"M 1159 525 \n",
"L 1159 -1331 \n",
"L 581 -1331 \n",
"L 581 3500 \n",
"L 1159 3500 \n",
"L 1159 2969 \n",
"Q 1341 3281 1617 3432 \n",
"Q 1894 3584 2278 3584 \n",
"Q 2916 3584 3314 3078 \n",
"Q 3713 2572 3713 1747 \n",
"Q 3713 922 3314 415 \n",
"Q 2916 -91 2278 -91 \n",
"Q 1894 -91 1617 61 \n",
"Q 1341 213 1159 525 \n",
"z\n",
"M 3116 1747 \n",
"Q 3116 2381 2855 2742 \n",
"Q 2594 3103 2138 3103 \n",
"Q 1681 3103 1420 2742 \n",
"Q 1159 2381 1159 1747 \n",
"Q 1159 1113 1420 752 \n",
"Q 1681 391 2138 391 \n",
"Q 2594 391 2855 752 \n",
"Q 3116 1113 3116 1747 \n",
"z\n",
"\" transform=\"scale(0.015625)\"/>\n",
" <path id=\"DejaVuSans-6f\" d=\"M 1959 3097 \n",
"Q 1497 3097 1228 2736 \n",
"Q 959 2375 959 1747 \n",
"Q 959 1119 1226 758 \n",
"Q 1494 397 1959 397 \n",
"Q 2419 397 2687 759 \n",
"Q 2956 1122 2956 1747 \n",
"Q 2956 2369 2687 2733 \n",
"Q 2419 3097 1959 3097 \n",
"z\n",
"M 1959 3584 \n",
"Q 2709 3584 3137 3096 \n",
"Q 3566 2609 3566 1747 \n",
"Q 3566 888 3137 398 \n",
"Q 2709 -91 1959 -91 \n",
"Q 1206 -91 779 398 \n",
"Q 353 888 353 1747 \n",
"Q 353 2609 779 3096 \n",
"Q 1206 3584 1959 3584 \n",
"z\n",
"\" transform=\"scale(0.015625)\"/>\n",
" <path id=\"DejaVuSans-63\" d=\"M 3122 3366 \n",
"L 3122 2828 \n",
"Q 2878 2963 2633 3030 \n",
"Q 2388 3097 2138 3097 \n",
"Q 1578 3097 1268 2742 \n",
"Q 959 2388 959 1747 \n",
"Q 959 1106 1268 751 \n",
"Q 1578 397 2138 397 \n",
"Q 2388 397 2633 464 \n",
"Q 2878 531 3122 666 \n",
"L 3122 134 \n",
"Q 2881 22 2623 -34 \n",
"Q 2366 -91 2075 -91 \n",
"Q 1284 -91 818 406 \n",
"Q 353 903 353 1747 \n",
"Q 353 2603 823 3093 \n",
"Q 1294 3584 2113 3584 \n",
"Q 2378 3584 2631 3529 \n",
"Q 2884 3475 3122 3366 \n",
"z\n",
"\" transform=\"scale(0.015625)\"/>\n",
" <path id=\"DejaVuSans-68\" d=\"M 3513 2113 \n",
"L 3513 0 \n",
"L 2938 0 \n",
"L 2938 2094 \n",
"Q 2938 2591 2744 2837 \n",
"Q 2550 3084 2163 3084 \n",
"Q 1697 3084 1428 2787 \n",
"Q 1159 2491 1159 1978 \n",
"L 1159 0 \n",
"L 581 0 \n",
"L 581 4863 \n",
"L 1159 4863 \n",
"L 1159 2956 \n",
"Q 1366 3272 1645 3428 \n",
"Q 1925 3584 2291 3584 \n",
"Q 2894 3584 3203 3211 \n",
"Q 3513 2838 3513 2113 \n",
"z\n",
"\" transform=\"scale(0.015625)\"/>\n",
" </defs>\n",
" <use xlink:href=\"#DejaVuSans-65\"/>\n",
" <use xlink:href=\"#DejaVuSans-70\" x=\"61.523438\"/>\n",
" <use xlink:href=\"#DejaVuSans-6f\" x=\"125\"/>\n",
" <use xlink:href=\"#DejaVuSans-63\" x=\"186.181641\"/>\n",
" <use xlink:href=\"#DejaVuSans-68\" x=\"241.162109\"/>\n",
" </g>\n",
" </g>\n",
" </g>\n",
" <g id=\"matplotlib.axis_2\">\n",
" <g id=\"ytick_1\">\n",
" <g id=\"line2d_11\">\n",
" <path d=\"M 30.103125 120.45 \n",
"L 225.403125 120.45 \n",
"\" clip-path=\"url(#p38f7277f50)\" style=\"fill: none; stroke: #b0b0b0; stroke-width: 0.8; stroke-linecap: square\"/>\n",
" </g>\n",
" <g id=\"line2d_12\">\n",
" <defs>\n",
" <path id=\"m0ca26dcbeb\" d=\"M 0 0 \n",
"L -3.5 0 \n",
"\" style=\"stroke: #000000; stroke-width: 0.8\"/>\n",
" </defs>\n",
" <g>\n",
" <use xlink:href=\"#m0ca26dcbeb\" x=\"30.103125\" y=\"120.45\" style=\"stroke: #000000; stroke-width: 0.8\"/>\n",
" </g>\n",
" </g>\n",
" <g id=\"text_7\">\n",
" <!-- 0.4 -->\n",
" <g transform=\"translate(7.2 124.249219)scale(0.1 -0.1)\">\n",
" <defs>\n",
" <path id=\"DejaVuSans-2e\" d=\"M 684 794 \n",
"L 1344 794 \n",
"L 1344 0 \n",
"L 684 0 \n",
"L 684 794 \n",
"z\n",
"\" transform=\"scale(0.015625)\"/>\n",
" </defs>\n",
" <use xlink:href=\"#DejaVuSans-30\"/>\n",
" <use xlink:href=\"#DejaVuSans-2e\" x=\"63.623047\"/>\n",
" <use xlink:href=\"#DejaVuSans-34\" x=\"95.410156\"/>\n",
" </g>\n",
" </g>\n",
" </g>\n",
" <g id=\"ytick_2\">\n",
" <g id=\"line2d_13\">\n",
" <path d=\"M 30.103125 75.15 \n",
"L 225.403125 75.15 \n",
"\" clip-path=\"url(#p38f7277f50)\" style=\"fill: none; stroke: #b0b0b0; stroke-width: 0.8; stroke-linecap: square\"/>\n",
" </g>\n",
" <g id=\"line2d_14\">\n",
" <g>\n",
" <use xlink:href=\"#m0ca26dcbeb\" x=\"30.103125\" y=\"75.15\" style=\"stroke: #000000; stroke-width: 0.8\"/>\n",
" </g>\n",
" </g>\n",
" <g id=\"text_8\">\n",
" <!-- 0.6 -->\n",
" <g transform=\"translate(7.2 78.949219)scale(0.1 -0.1)\">\n",
" <use xlink:href=\"#DejaVuSans-30\"/>\n",
" <use xlink:href=\"#DejaVuSans-2e\" x=\"63.623047\"/>\n",
" <use xlink:href=\"#DejaVuSans-36\" x=\"95.410156\"/>\n",
" </g>\n",
" </g>\n",
" </g>\n",
" <g id=\"ytick_3\">\n",
" <g id=\"line2d_15\">\n",
" <path d=\"M 30.103125 29.85 \n",
"L 225.403125 29.85 \n",
"\" clip-path=\"url(#p38f7277f50)\" style=\"fill: none; stroke: #b0b0b0; stroke-width: 0.8; stroke-linecap: square\"/>\n",
" </g>\n",
" <g id=\"line2d_16\">\n",
" <g>\n",
" <use xlink:href=\"#m0ca26dcbeb\" x=\"30.103125\" y=\"29.85\" style=\"stroke: #000000; stroke-width: 0.8\"/>\n",
" </g>\n",
" </g>\n",
" <g id=\"text_9\">\n",
" <!-- 0.8 -->\n",
" <g transform=\"translate(7.2 33.649219)scale(0.1 -0.1)\">\n",
" <use xlink:href=\"#DejaVuSans-30\"/>\n",
" <use xlink:href=\"#DejaVuSans-2e\" x=\"63.623047\"/>\n",
" <use xlink:href=\"#DejaVuSans-38\" x=\"95.410156\"/>\n",
" </g>\n",
" </g>\n",
" </g>\n",
" </g>\n",
" <g id=\"line2d_17\">\n",
" <path d=\"M 35.272884 -1 \n",
"L 51.803125 75.61021 \n",
"L 73.503125 93.672344 \n",
"L 95.203125 102.778348 \n",
"L 116.903125 107.632437 \n",
"L 138.603125 112.487156 \n",
"L 160.303125 116.4354 \n",
"L 182.003125 119.040329 \n",
"L 203.703125 121.424263 \n",
"L 225.403125 124.527028 \n",
"\" clip-path=\"url(#p38f7277f50)\" style=\"fill: none; stroke: #1f77b4; stroke-width: 1.5; stroke-linecap: square\"/>\n",
" </g>\n",
" <g id=\"line2d_18\">\n",
" <path d=\"M 30.103125 65.6219 \n",
"L 51.803125 32.179175 \n",
"L 73.503125 25.7881 \n",
"L 95.203125 22.432125 \n",
"L 116.903125 21.005175 \n",
"L 138.603125 18.959125 \n",
"L 160.303125 18.0418 \n",
"L 182.003125 17.124475 \n",
"L 203.703125 16.0939 \n",
"L 225.403125 15.08975 \n",
"\" clip-path=\"url(#p38f7277f50)\" style=\"fill: none; stroke-dasharray: 5.55,2.4; stroke-dashoffset: 0; stroke: #bf00bf; stroke-width: 1.5\"/>\n",
" </g>\n",
" <g id=\"line2d_19\">\n",
" <path d=\"M 30.103125 41.6733 \n",
"L 51.803125 32.77185 \n",
"L 73.503125 25.11615 \n",
"L 95.203125 23.84775 \n",
"L 116.903125 27.3585 \n",
"L 138.603125 22.5567 \n",
"L 160.303125 23.84775 \n",
"L 182.003125 19.49895 \n",
"L 203.703125 22.7832 \n",
"L 225.403125 21.1977 \n",
"\" clip-path=\"url(#p38f7277f50)\" style=\"fill: none; stroke-dasharray: 9.6,2.4,1.5,2.4; stroke-dashoffset: 0; stroke: #008000; stroke-width: 1.5\"/>\n",
" </g>\n",
" <g id=\"patch_3\">\n",
" <path d=\"M 30.103125 143.1 \n",
"L 30.103125 7.2 \n",
"\" style=\"fill: none; stroke: #000000; stroke-width: 0.8; stroke-linejoin: miter; stroke-linecap: square\"/>\n",
" </g>\n",
" <g id=\"patch_4\">\n",
" <path d=\"M 225.403125 143.1 \n",
"L 225.403125 7.2 \n",
"\" style=\"fill: none; stroke: #000000; stroke-width: 0.8; stroke-linejoin: miter; stroke-linecap: square\"/>\n",
" </g>\n",
" <g id=\"patch_5\">\n",
" <path d=\"M 30.103125 143.1 \n",
"L 225.403125 143.1 \n",
"\" style=\"fill: none; stroke: #000000; stroke-width: 0.8; stroke-linejoin: miter; stroke-linecap: square\"/>\n",
" </g>\n",
" <g id=\"patch_6\">\n",
" <path d=\"M 30.103125 7.2 \n",
"L 225.403125 7.2 \n",
"\" style=\"fill: none; stroke: #000000; stroke-width: 0.8; stroke-linejoin: miter; stroke-linecap: square\"/>\n",
" </g>\n",
" <g id=\"legend_1\">\n",
" <g id=\"patch_7\">\n",
" <path d=\"M 140.634375 98.667187 \n",
"L 218.403125 98.667187 \n",
"Q 220.403125 98.667187 220.403125 96.667187 \n",
"L 220.403125 53.632812 \n",
"Q 220.403125 51.632812 218.403125 51.632812 \n",
"L 140.634375 51.632812 \n",
"Q 138.634375 51.632812 138.634375 53.632812 \n",
"L 138.634375 96.667187 \n",
"Q 138.634375 98.667187 140.634375 98.667187 \n",
"z\n",
"\" style=\"fill: #ffffff; opacity: 0.8; stroke: #cccccc; stroke-linejoin: miter\"/>\n",
" </g>\n",
" <g id=\"line2d_20\">\n",
" <path d=\"M 142.634375 59.73125 \n",
"L 152.634375 59.73125 \n",
"L 162.634375 59.73125 \n",
"\" style=\"fill: none; stroke: #1f77b4; stroke-width: 1.5; stroke-linecap: square\"/>\n",
" </g>\n",
" <g id=\"text_10\">\n",
" <!-- train loss -->\n",
" <g transform=\"translate(170.634375 63.23125)scale(0.1 -0.1)\">\n",
" <defs>\n",
" <path id=\"DejaVuSans-74\" d=\"M 1172 4494 \n",
"L 1172 3500 \n",
"L 2356 3500 \n",
"L 2356 3053 \n",
"L 1172 3053 \n",
"L 1172 1153 \n",
"Q 1172 725 1289 603 \n",
"Q 1406 481 1766 481 \n",
"L 2356 481 \n",
"L 2356 0 \n",
"L 1766 0 \n",
"Q 1100 0 847 248 \n",
"Q 594 497 594 1153 \n",
"L 594 3053 \n",
"L 172 3053 \n",
"L 172 3500 \n",
"L 594 3500 \n",
"L 594 4494 \n",
"L 1172 4494 \n",
"z\n",
"\" transform=\"scale(0.015625)\"/>\n",
" <path id=\"DejaVuSans-72\" d=\"M 2631 2963 \n",
"Q 2534 3019 2420 3045 \n",
"Q 2306 3072 2169 3072 \n",
"Q 1681 3072 1420 2755 \n",
"Q 1159 2438 1159 1844 \n",
"L 1159 0 \n",
"L 581 0 \n",
"L 581 3500 \n",
"L 1159 3500 \n",
"L 1159 2956 \n",
"Q 1341 3275 1631 3429 \n",
"Q 1922 3584 2338 3584 \n",
"Q 2397 3584 2469 3576 \n",
"Q 2541 3569 2628 3553 \n",
"L 2631 2963 \n",
"z\n",
"\" transform=\"scale(0.015625)\"/>\n",
" <path id=\"DejaVuSans-61\" d=\"M 2194 1759 \n",
"Q 1497 1759 1228 1600 \n",
"Q 959 1441 959 1056 \n",
"Q 959 750 1161 570 \n",
"Q 1363 391 1709 391 \n",
"Q 2188 391 2477 730 \n",
"Q 2766 1069 2766 1631 \n",
"L 2766 1759 \n",
"L 2194 1759 \n",
"z\n",
"M 3341 1997 \n",
"L 3341 0 \n",
"L 2766 0 \n",
"L 2766 531 \n",
"Q 2569 213 2275 61 \n",
"Q 1981 -91 1556 -91 \n",
"Q 1019 -91 701 211 \n",
"Q 384 513 384 1019 \n",
"Q 384 1609 779 1909 \n",
"Q 1175 2209 1959 2209 \n",
"L 2766 2209 \n",
"L 2766 2266 \n",
"Q 2766 2663 2505 2880 \n",
"Q 2244 3097 1772 3097 \n",
"Q 1472 3097 1187 3025 \n",
"Q 903 2953 641 2809 \n",
"L 641 3341 \n",
"Q 956 3463 1253 3523 \n",
"Q 1550 3584 1831 3584 \n",
"Q 2591 3584 2966 3190 \n",
"Q 3341 2797 3341 1997 \n",
"z\n",
"\" transform=\"scale(0.015625)\"/>\n",
" <path id=\"DejaVuSans-69\" d=\"M 603 3500 \n",
"L 1178 3500 \n",
"L 1178 0 \n",
"L 603 0 \n",
"L 603 3500 \n",
"z\n",
"M 603 4863 \n",
"L 1178 4863 \n",
"L 1178 4134 \n",
"L 603 4134 \n",
"L 603 4863 \n",
"z\n",
"\" transform=\"scale(0.015625)\"/>\n",
" <path id=\"DejaVuSans-6e\" d=\"M 3513 2113 \n",
"L 3513 0 \n",
"L 2938 0 \n",
"L 2938 2094 \n",
"Q 2938 2591 2744 2837 \n",
"Q 2550 3084 2163 3084 \n",
"Q 1697 3084 1428 2787 \n",
"Q 1159 2491 1159 1978 \n",
"L 1159 0 \n",
"L 581 0 \n",
"L 581 3500 \n",
"L 1159 3500 \n",
"L 1159 2956 \n",
"Q 1366 3272 1645 3428 \n",
"Q 1925 3584 2291 3584 \n",
"Q 2894 3584 3203 3211 \n",
"Q 3513 2838 3513 2113 \n",
"z\n",
"\" transform=\"scale(0.015625)\"/>\n",
" <path id=\"DejaVuSans-20\" transform=\"scale(0.015625)\"/>\n",
" <path id=\"DejaVuSans-6c\" d=\"M 603 4863 \n",
"L 1178 4863 \n",
"L 1178 0 \n",
"L 603 0 \n",
"L 603 4863 \n",
"z\n",
"\" transform=\"scale(0.015625)\"/>\n",
" <path id=\"DejaVuSans-73\" d=\"M 2834 3397 \n",
"L 2834 2853 \n",
"Q 2591 2978 2328 3040 \n",
"Q 2066 3103 1784 3103 \n",
"Q 1356 3103 1142 2972 \n",
"Q 928 2841 928 2578 \n",
"Q 928 2378 1081 2264 \n",
"Q 1234 2150 1697 2047 \n",
"L 1894 2003 \n",
"Q 2506 1872 2764 1633 \n",
"Q 3022 1394 3022 966 \n",
"Q 3022 478 2636 193 \n",
"Q 2250 -91 1575 -91 \n",
"Q 1294 -91 989 -36 \n",
"Q 684 19 347 128 \n",
"L 347 722 \n",
"Q 666 556 975 473 \n",
"Q 1284 391 1588 391 \n",
"Q 1994 391 2212 530 \n",
"Q 2431 669 2431 922 \n",
"Q 2431 1156 2273 1281 \n",
"Q 2116 1406 1581 1522 \n",
"L 1381 1569 \n",
"Q 847 1681 609 1914 \n",
"Q 372 2147 372 2553 \n",
"Q 372 3047 722 3315 \n",
"Q 1072 3584 1716 3584 \n",
"Q 2034 3584 2315 3537 \n",
"Q 2597 3491 2834 3397 \n",
"z\n",
"\" transform=\"scale(0.015625)\"/>\n",
" </defs>\n",
" <use xlink:href=\"#DejaVuSans-74\"/>\n",
" <use xlink:href=\"#DejaVuSans-72\" x=\"39.208984\"/>\n",
" <use xlink:href=\"#DejaVuSans-61\" x=\"80.322266\"/>\n",
" <use xlink:href=\"#DejaVuSans-69\" x=\"141.601562\"/>\n",
" <use xlink:href=\"#DejaVuSans-6e\" x=\"169.384766\"/>\n",
" <use xlink:href=\"#DejaVuSans-20\" x=\"232.763672\"/>\n",
" <use xlink:href=\"#DejaVuSans-6c\" x=\"264.550781\"/>\n",
" <use xlink:href=\"#DejaVuSans-6f\" x=\"292.333984\"/>\n",
" <use xlink:href=\"#DejaVuSans-73\" x=\"353.515625\"/>\n",
" <use xlink:href=\"#DejaVuSans-73\" x=\"405.615234\"/>\n",
" </g>\n",
" </g>\n",
" <g id=\"line2d_21\">\n",
" <path d=\"M 142.634375 74.409375 \n",
"L 152.634375 74.409375 \n",
"L 162.634375 74.409375 \n",
"\" style=\"fill: none; stroke-dasharray: 5.55,2.4; stroke-dashoffset: 0; stroke: #bf00bf; stroke-width: 1.5\"/>\n",
" </g>\n",
" <g id=\"text_11\">\n",
" <!-- train acc -->\n",
" <g transform=\"translate(170.634375 77.909375)scale(0.1 -0.1)\">\n",
" <use xlink:href=\"#DejaVuSans-74\"/>\n",
" <use xlink:href=\"#DejaVuSans-72\" x=\"39.208984\"/>\n",
" <use xlink:href=\"#DejaVuSans-61\" x=\"80.322266\"/>\n",
" <use xlink:href=\"#DejaVuSans-69\" x=\"141.601562\"/>\n",
" <use xlink:href=\"#DejaVuSans-6e\" x=\"169.384766\"/>\n",
" <use xlink:href=\"#DejaVuSans-20\" x=\"232.763672\"/>\n",
" <use xlink:href=\"#DejaVuSans-61\" x=\"264.550781\"/>\n",
" <use xlink:href=\"#DejaVuSans-63\" x=\"325.830078\"/>\n",
" <use xlink:href=\"#DejaVuSans-63\" x=\"380.810547\"/>\n",
" </g>\n",
" </g>\n",
" <g id=\"line2d_22\">\n",
" <path d=\"M 142.634375 89.0875 \n",
"L 152.634375 89.0875 \n",
"L 162.634375 89.0875 \n",
"\" style=\"fill: none; stroke-dasharray: 9.6,2.4,1.5,2.4; stroke-dashoffset: 0; stroke: #008000; stroke-width: 1.5\"/>\n",
" </g>\n",
" <g id=\"text_12\">\n",
" <!-- test acc -->\n",
" <g transform=\"translate(170.634375 92.5875)scale(0.1 -0.1)\">\n",
" <use xlink:href=\"#DejaVuSans-74\"/>\n",
" <use xlink:href=\"#DejaVuSans-65\" x=\"39.208984\"/>\n",
" <use xlink:href=\"#DejaVuSans-73\" x=\"100.732422\"/>\n",
" <use xlink:href=\"#DejaVuSans-74\" x=\"152.832031\"/>\n",
" <use xlink:href=\"#DejaVuSans-20\" x=\"192.041016\"/>\n",
" <use xlink:href=\"#DejaVuSans-61\" x=\"223.828125\"/>\n",
" <use xlink:href=\"#DejaVuSans-63\" x=\"285.107422\"/>\n",
" <use xlink:href=\"#DejaVuSans-63\" x=\"340.087891\"/>\n",
" </g>\n",
" </g>\n",
" </g>\n",
" </g>\n",
" </g>\n",
" <defs>\n",
" <clipPath id=\"p38f7277f50\">\n",
" <rect x=\"30.103125\" y=\"7.2\" width=\"195.3\" height=\"135.9\"/>\n",
" </clipPath>\n",
" </defs>\n",
"</svg>\n"
],
"text/plain": [
"<Figure size 252x180 with 1 Axes>"
]
},
"metadata": {
"needs_background": "light"
},
"output_type": "display_data"
}
],
"source": [
"train_iter, test_iter = d2l.load_data_fashion_mnist(batch_size)\n",
"d2l.train_ch3(net, train_iter, test_iter, loss, num_epochs, trainer)"
]
},
{
"cell_type": "markdown",
"id": "9b636c57",
"metadata": {
"origin_pos": 16
},
"source": [
"## 小结\n",
"\n",
"* 我们可以使用高级API更简洁地实现多层感知机。\n",
"* 对于相同的分类问题，多层感知机的实现与softmax回归的实现相同，只是多层感知机的实现里增加了带有激活函数的隐藏层。\n",
"\n",
"## 练习\n",
"\n",
"1. 尝试添加不同数量的隐藏层（也可以修改学习率），怎么样设置效果最好？\n",
"1. 尝试不同的激活函数，哪个效果最好？\n",
"1. 尝试不同的方案来初始化权重，什么方法效果最好？\n"
]
},
{
"cell_type": "markdown",
"id": "36201fb3",
"metadata": {
"origin_pos": 18,
"tab": [
"pytorch"
]
},
"source": [
"[Discussions](https://discuss.d2l.ai/t/1802)\n"
]
}
],
"metadata": {
"language_info": {
"name": "python"
},
"required_libs": []
},
"nbformat": 4,
"nbformat_minor": 5
}
\ No newline at end of file
前馈神经网络shr/实例代码/multilayer-perceptrons_test.ipynb 0 → 100644
{
{
"cells": [
{
"cell_type": "markdown",
"id": "59b31683",
"metadata": {
"origin_pos": 0
},
"source": [
"# 多层感知机的从零开始实现\n",
":label:`sec_mlp_scratch`\n",
"\n",
"我们已经在 :numref:`sec_mlp`中描述了多层感知机（MLP），\n",
"现在让我们尝试自己实现一个多层感知机。\n",
"为了与之前softmax回归（ :numref:`sec_softmax_scratch` ）\n",
"获得的结果进行比较，\n",
"我们将继续使用Fashion-MNIST图像分类数据集\n",
"（ :numref:`sec_fashion_mnist`）。\n"
]
},
{
"cell_type": "code",
"execution_count": 1,
"id": "ffbb0fc1",
"metadata": {
"execution": {
"iopub.execute_input": "2023-08-18T06:59:21.394152Z",
"iopub.status.busy": "2023-08-18T06:59:21.393407Z",
"iopub.status.idle": "2023-08-18T06:59:24.364157Z",
"shell.execute_reply": "2023-08-18T06:59:24.362977Z"
},
"origin_pos": 2,
"tab": [
"pytorch"
]
},
"outputs": [],
"source": [
"import torch #torch 包含了多维张量的数据结构以及基于其上的多种数学操作\n",
"from torch import nn #torch.nn 是pytorch 的一种API工具，该工具是为了快速搭建神经网络任务而创建的。\n",
"from d2l import torch as d2l"
]
},
{
"cell_type": "code",
"execution_count": 2,
"id": "0be61c4f",
"metadata": {
"execution": {
"iopub.execute_input": "2023-08-18T06:59:24.369567Z",
"iopub.status.busy": "2023-08-18T06:59:24.368990Z",
"iopub.status.idle": "2023-08-18T06:59:24.501326Z",
"shell.execute_reply": "2023-08-18T06:59:24.500151Z"
},
"origin_pos": 5,
"tab": [
"pytorch"
]
},
"outputs": [],
"source": [
"batch_size = 256 #批量：每次训练神经网络时，同时输入模型的训练样本数量\n",
"train_iter, test_iter = d2l.load_data_fashion_mnist(batch_size) "
]
},
{
"cell_type": "markdown",
"id": "8236e2cd",
"metadata": {
"origin_pos": 6
},
"source": [
"## 初始化模型参数\n",
"\n",
"Fashion-MNIST中的每个图像由\n",
"$28 \\times 28 = 784$个灰度像素值组成。\n",
"所有图像共分为10个类别。\n",
"忽略像素之间的空间结构，\n",
"我们可以将每个图像视为具有784个输入特征\n",
"和10个类的简单分类数据集。\n",
"首先，我们将[**实现一个具有单隐藏层的多层感知机，\n",
"它包含256个隐藏单元**]。\n",
"注意，我们可以将这两个变量都视为超参数。\n",
"通常，我们选择2的若干次幂作为层的宽度。\n",
"因为内存在硬件中的分配和寻址方式，这么做往往可以在计算上更高效。\n",
"\n",
"我们用几个张量来表示我们的参数。\n",
"注意，对于每一层我们都要记录一个权重矩阵和一个偏置向量。\n",
"同时，我们要为损失关于这些参数的梯度分配内存。\n"
]
},
{
"cell_type": "code",
"execution_count": 3,
"id": "7730f280",
"metadata": {
"execution": {
"iopub.execute_input": "2023-08-18T06:59:24.508163Z",
"iopub.status.busy": "2023-08-18T06:59:24.506257Z",
"iopub.status.idle": "2023-08-18T06:59:24.520861Z",
"shell.execute_reply": "2023-08-18T06:59:24.519861Z"
},
"origin_pos": 8,
"tab": [
"pytorch"
]
},
"outputs": [],
"source": [
"num_inputs, num_outputs, num_hiddens = 784, 10, 256\n",
"\n",
"W1 = nn.Parameter(torch.randn(\n",
" num_inputs, num_hiddens, requires_grad=True) * 0.01)\n",
"b1 = nn.Parameter(torch.zeros(num_hiddens, requires_grad=True))\n",
"W2 = nn.Parameter(torch.randn(\n",
" num_hiddens, num_outputs, requires_grad=True) * 0.01)\n",
"b2 = nn.Parameter(torch.zeros(num_outputs, requires_grad=True))\n",
"\n",
"params = [W1, b1, W2, b2]"
]
},
{
"cell_type": "markdown",
"id": "2700dfe8",
"metadata": {
"origin_pos": 11
},
"source": [
"## 激活函数\n",
"\n",
"为了确保我们对模型的细节了如指掌，\n",
"我们将[**实现ReLU激活函数**]，\n",
"而不是直接调用内置的`relu`函数。\n"
]
},
{
"cell_type": "code",
"execution_count": 4,
"id": "5f46a813",
"metadata": {
"execution": {
"iopub.execute_input": "2023-08-18T06:59:24.528151Z",
"iopub.status.busy": "2023-08-18T06:59:24.526356Z",
"iopub.status.idle": "2023-08-18T06:59:24.533695Z",
"shell.execute_reply": "2023-08-18T06:59:24.532654Z"
},
"origin_pos": 13,
"tab": [
"pytorch"
]
},
"outputs": [],
"source": [
"def relu(X):\n",
" a = torch.zeros_like(X)\n",
" return torch.max(X, a)"
]
},
{
"cell_type": "markdown",
"id": "741dbe39",
"metadata": {
"origin_pos": 16
},
"source": [
"## 模型\n",
"\n",
"因为我们忽略了空间结构，\n",
"所以我们使用`reshape`将每个二维图像转换为一个长度为`num_inputs`的向量。\n",
"只需几行代码就可以(**实现我们的模型**)。\n"
]
},
{
"cell_type": "code",
"execution_count": 5,
"id": "b3d9923a",
"metadata": {
"execution": {
"iopub.execute_input": "2023-08-18T06:59:24.541482Z",
"iopub.status.busy": "2023-08-18T06:59:24.539621Z",
"iopub.status.idle": "2023-08-18T06:59:24.547435Z",
"shell.execute_reply": "2023-08-18T06:59:24.546468Z"
},
"origin_pos": 18,
"tab": [
"pytorch"
]
},
"outputs": [],
"source": [
"def net(X):\n",
" X = X.reshape((-1, num_inputs)) # -1表示与原数组行一致，即为batch_size\n",
" H = relu(X@W1 + b1) # 这里“@”代表矩阵乘法\n",
" return (H@W2 + b2)"
]
},
{
"cell_type": "markdown",
"id": "bd600c14",
"metadata": {
"origin_pos": 21
},
"source": [
"## 损失函数\n",
"\n",
"在这里我们直接使用高级API中的内置函数来计算softmax和交叉熵损失。\n",
"\n"
]
},
{
"cell_type": "code",
"execution_count": 6,
"id": "f55fe0ea",
"metadata": {
"execution": {
"iopub.execute_input": "2023-08-18T06:59:24.554675Z",
"iopub.status.busy": "2023-08-18T06:59:24.552824Z",
"iopub.status.idle": "2023-08-18T06:59:24.560084Z",
"shell.execute_reply": "2023-08-18T06:59:24.559049Z"
},
"origin_pos": 23,
"tab": [
"pytorch"
]
},
"outputs": [],
"source": [
"loss = nn.CrossEntropyLoss(reduction='none')"
]
},
{
"cell_type": "markdown",
"id": "b3a03c3a",
"metadata": {
"origin_pos": 25
},
"source": [
"## 训练\n",
"\n",
"幸运的是，[**多层感知机的训练过程与softmax回归的训练过程完全相同**]。\n",
"可以直接调用`d2l`包的`train_ch3`函数（参见 :numref:`sec_softmax_scratch` ），\n",
"将迭代周期数设置为10，并将学习率设置为0.1.\n"
]
},
{
"cell_type": "code",
"execution_count": 7,
"id": "c83cc0c7",
"metadata": {
"execution": {
"iopub.execute_input": "2023-08-18T06:59:24.567796Z",
"iopub.status.busy": "2023-08-18T06:59:24.566005Z",
"iopub.status.idle": "2023-08-18T07:00:19.750339Z",
"shell.execute_reply": "2023-08-18T07:00:19.748990Z"
},
"origin_pos": 27,
"tab": [
"pytorch"
]
},
"outputs": [
{
"data": {
"image/svg+xml": [
"<?xml version=\"1.0\" encoding=\"utf-8\" standalone=\"no\"?>\n",
"<!DOCTYPE svg PUBLIC \"-//W3C//DTD SVG 1.1//EN\"\n",
" \"http://www.w3.org/Graphics/SVG/1.1/DTD/svg11.dtd\">\n",
"<svg xmlns:xlink=\"http://www.w3.org/1999/xlink\" width=\"238.965625pt\" height=\"183.35625pt\" viewBox=\"0 0 238.965625 183.35625\" xmlns=\"http://www.w3.org/2000/svg\" version=\"1.1\">\n",
" <metadata>\n",
" <rdf:RDF xmlns:dc=\"http://purl.org/dc/elements/1.1/\" xmlns:cc=\"http://creativecommons.org/ns#\" xmlns:rdf=\"http://www.w3.org/1999/02/22-rdf-syntax-ns#\">\n",
" <cc:Work>\n",
" <dc:type rdf:resource=\"http://purl.org/dc/dcmitype/StillImage\"/>\n",
" <dc:date>2023-10-31T20:43:10.488883</dc:date>\n",
" <dc:format>image/svg+xml</dc:format>\n",
" <dc:creator>\n",
" <cc:Agent>\n",
" <dc:title>Matplotlib v3.5.1, https://matplotlib.org/</dc:title>\n",
" </cc:Agent>\n",
" </dc:creator>\n",
" </cc:Work>\n",
" </rdf:RDF>\n",
" </metadata>\n",
" <defs>\n",
" <style type=\"text/css\">*{stroke-linejoin: round; stroke-linecap: butt}</style>\n",
" </defs>\n",
" <g id=\"figure_1\">\n",
" <g id=\"patch_1\">\n",
" <path d=\"M 0 183.35625 \n",
"L 238.965625 183.35625 \n",
"L 238.965625 0 \n",
"L 0 0 \n",
"z\n",
"\" style=\"fill: #ffffff\"/>\n",
" </g>\n",
" <g id=\"axes_1\">\n",
" <g id=\"patch_2\">\n",
" <path d=\"M 30.103125 145.8 \n",
"L 225.403125 145.8 \n",
"L 225.403125 7.2 \n",
"L 30.103125 7.2 \n",
"z\n",
"\" style=\"fill: #ffffff\"/>\n",
" </g>\n",
" <g id=\"matplotlib.axis_1\">\n",
" <g id=\"xtick_1\">\n",
" <g id=\"line2d_1\">\n",
" <path d=\"M 51.803125 145.8 \n",
"L 51.803125 7.2 \n",
"\" clip-path=\"url(#p85ad12d352)\" style=\"fill: none; stroke: #b0b0b0; stroke-width: 0.8; stroke-linecap: square\"/>\n",
" </g>\n",
" <g id=\"line2d_2\">\n",
" <defs>\n",
" <path id=\"m4ae42fe94b\" d=\"M 0 0 \n",
"L 0 3.5 \n",
"\" style=\"stroke: #000000; stroke-width: 0.8\"/>\n",
" </defs>\n",
" <g>\n",
" <use xlink:href=\"#m4ae42fe94b\" x=\"51.803125\" y=\"145.8\" style=\"stroke: #000000; stroke-width: 0.8\"/>\n",
" </g>\n",
" </g>\n",
" <g id=\"text_1\">\n",
" <!-- 2 -->\n",
" <g transform=\"translate(48.621875 160.398438)scale(0.1 -0.1)\">\n",
" <defs>\n",
" <path id=\"DejaVuSans-32\" d=\"M 1228 531 \n",
"L 3431 531 \n",
"L 3431 0 \n",
"L 469 0 \n",
"L 469 531 \n",
"Q 828 903 1448 1529 \n",
"Q 2069 2156 2228 2338 \n",
"Q 2531 2678 2651 2914 \n",
"Q 2772 3150 2772 3378 \n",
"Q 2772 3750 2511 3984 \n",
"Q 2250 4219 1831 4219 \n",
"Q 1534 4219 1204 4116 \n",
"Q 875 4013 500 3803 \n",
"L 500 4441 \n",
"Q 881 4594 1212 4672 \n",
"Q 1544 4750 1819 4750 \n",
"Q 2544 4750 2975 4387 \n",
"Q 3406 4025 3406 3419 \n",
"Q 3406 3131 3298 2873 \n",
"Q 3191 2616 2906 2266 \n",
"Q 2828 2175 2409 1742 \n",
"Q 1991 1309 1228 531 \n",
"z\n",
"\" transform=\"scale(0.015625)\"/>\n",
" </defs>\n",
" <use xlink:href=\"#DejaVuSans-32\"/>\n",
" </g>\n",
" </g>\n",
" </g>\n",
" <g id=\"xtick_2\">\n",
" <g id=\"line2d_3\">\n",
" <path d=\"M 95.203125 145.8 \n",
"L 95.203125 7.2 \n",
"\" clip-path=\"url(#p85ad12d352)\" style=\"fill: none; stroke: #b0b0b0; stroke-width: 0.8; stroke-linecap: square\"/>\n",
" </g>\n",
" <g id=\"line2d_4\">\n",
" <g>\n",
" <use xlink:href=\"#m4ae42fe94b\" x=\"95.203125\" y=\"145.8\" style=\"stroke: #000000; stroke-width: 0.8\"/>\n",
" </g>\n",
" </g>\n",
" <g id=\"text_2\">\n",
" <!-- 4 -->\n",
" <g transform=\"translate(92.021875 160.398438)scale(0.1 -0.1)\">\n",
" <defs>\n",
" <path id=\"DejaVuSans-34\" d=\"M 2419 4116 \n",
"L 825 1625 \n",
"L 2419 1625 \n",
"L 2419 4116 \n",
"z\n",
"M 2253 4666 \n",
"L 3047 4666 \n",
"L 3047 1625 \n",
"L 3713 1625 \n",
"L 3713 1100 \n",
"L 3047 1100 \n",
"L 3047 0 \n",
"L 2419 0 \n",
"L 2419 1100 \n",
"L 313 1100 \n",
"L 313 1709 \n",
"L 2253 4666 \n",
"z\n",
"\" transform=\"scale(0.015625)\"/>\n",
" </defs>\n",
" <use xlink:href=\"#DejaVuSans-34\"/>\n",
" </g>\n",
" </g>\n",
" </g>\n",
" <g id=\"xtick_3\">\n",
" <g id=\"line2d_5\">\n",
" <path d=\"M 138.603125 145.8 \n",
"L 138.603125 7.2 \n",
"\" clip-path=\"url(#p85ad12d352)\" style=\"fill: none; stroke: #b0b0b0; stroke-width: 0.8; stroke-linecap: square\"/>\n",
" </g>\n",
" <g id=\"line2d_6\">\n",
" <g>\n",
" <use xlink:href=\"#m4ae42fe94b\" x=\"138.603125\" y=\"145.8\" style=\"stroke: #000000; stroke-width: 0.8\"/>\n",
" </g>\n",
" </g>\n",
" <g id=\"text_3\">\n",
" <!-- 6 -->\n",
" <g transform=\"translate(135.421875 160.398438)scale(0.1 -0.1)\">\n",
" <defs>\n",
" <path id=\"DejaVuSans-36\" d=\"M 2113 2584 \n",
"Q 1688 2584 1439 2293 \n",
"Q 1191 2003 1191 1497 \n",
"Q 1191 994 1439 701 \n",
"Q 1688 409 2113 409 \n",
"Q 2538 409 2786 701 \n",
"Q 3034 994 3034 1497 \n",
"Q 3034 2003 2786 2293 \n",
"Q 2538 2584 2113 2584 \n",
"z\n",
"M 3366 4563 \n",
"L 3366 3988 \n",
"Q 3128 4100 2886 4159 \n",
"Q 2644 4219 2406 4219 \n",
"Q 1781 4219 1451 3797 \n",
"Q 1122 3375 1075 2522 \n",
"Q 1259 2794 1537 2939 \n",
"Q 1816 3084 2150 3084 \n",
"Q 2853 3084 3261 2657 \n",
"Q 3669 2231 3669 1497 \n",
"Q 3669 778 3244 343 \n",
"Q 2819 -91 2113 -91 \n",
"Q 1303 -91 875 529 \n",
"Q 447 1150 447 2328 \n",
"Q 447 3434 972 4092 \n",
"Q 1497 4750 2381 4750 \n",
"Q 2619 4750 2861 4703 \n",
"Q 3103 4656 3366 4563 \n",
"z\n",
"\" transform=\"scale(0.015625)\"/>\n",
" </defs>\n",
" <use xlink:href=\"#DejaVuSans-36\"/>\n",
" </g>\n",
" </g>\n",
" </g>\n",
" <g id=\"xtick_4\">\n",
" <g id=\"line2d_7\">\n",
" <path d=\"M 182.003125 145.8 \n",
"L 182.003125 7.2 \n",
"\" clip-path=\"url(#p85ad12d352)\" style=\"fill: none; stroke: #b0b0b0; stroke-width: 0.8; stroke-linecap: square\"/>\n",
" </g>\n",
" <g id=\"line2d_8\">\n",
" <g>\n",
" <use xlink:href=\"#m4ae42fe94b\" x=\"182.003125\" y=\"145.8\" style=\"stroke: #000000; stroke-width: 0.8\"/>\n",
" </g>\n",
" </g>\n",
" <g id=\"text_4\">\n",
" <!-- 8 -->\n",
" <g transform=\"translate(178.821875 160.398438)scale(0.1 -0.1)\">\n",
" <defs>\n",
" <path id=\"DejaVuSans-38\" d=\"M 2034 2216 \n",
"Q 1584 2216 1326 1975 \n",
"Q 1069 1734 1069 1313 \n",
"Q 1069 891 1326 650 \n",
"Q 1584 409 2034 409 \n",
"Q 2484 409 2743 651 \n",
"Q 3003 894 3003 1313 \n",
"Q 3003 1734 2745 1975 \n",
"Q 2488 2216 2034 2216 \n",
"z\n",
"M 1403 2484 \n",
"Q 997 2584 770 2862 \n",
"Q 544 3141 544 3541 \n",
"Q 544 4100 942 4425 \n",
"Q 1341 4750 2034 4750 \n",
"Q 2731 4750 3128 4425 \n",
"Q 3525 4100 3525 3541 \n",
"Q 3525 3141 3298 2862 \n",
"Q 3072 2584 2669 2484 \n",
"Q 3125 2378 3379 2068 \n",
"Q 3634 1759 3634 1313 \n",
"Q 3634 634 3220 271 \n",
"Q 2806 -91 2034 -91 \n",
"Q 1263 -91 848 271 \n",
"Q 434 634 434 1313 \n",
"Q 434 1759 690 2068 \n",
"Q 947 2378 1403 2484 \n",
"z\n",
"M 1172 3481 \n",
"Q 1172 3119 1398 2916 \n",
"Q 1625 2713 2034 2713 \n",
"Q 2441 2713 2670 2916 \n",
"Q 2900 3119 2900 3481 \n",
"Q 2900 3844 2670 4047 \n",
"Q 2441 4250 2034 4250 \n",
"Q 1625 4250 1398 4047 \n",
"Q 1172 3844 1172 3481 \n",
"z\n",
"\" transform=\"scale(0.015625)\"/>\n",
" </defs>\n",
" <use xlink:href=\"#DejaVuSans-38\"/>\n",
" </g>\n",
" </g>\n",
" </g>\n",
" <g id=\"xtick_5\">\n",
" <g id=\"line2d_9\">\n",
" <path d=\"M 225.403125 145.8 \n",
"L 225.403125 7.2 \n",
"\" clip-path=\"url(#p85ad12d352)\" style=\"fill: none; stroke: #b0b0b0; stroke-width: 0.8; stroke-linecap: square\"/>\n",
" </g>\n",
" <g id=\"line2d_10\">\n",
" <g>\n",
" <use xlink:href=\"#m4ae42fe94b\" x=\"225.403125\" y=\"145.8\" style=\"stroke: #000000; stroke-width: 0.8\"/>\n",
" </g>\n",
" </g>\n",
" <g id=\"text_5\">\n",
" <!-- 10 -->\n",
" <g transform=\"translate(219.040625 160.398438)scale(0.1 -0.1)\">\n",
" <defs>\n",
" <path id=\"DejaVuSans-31\" d=\"M 794 531 \n",
"L 1825 531 \n",
"L 1825 4091 \n",
"L 703 3866 \n",
"L 703 4441 \n",
"L 1819 4666 \n",
"L 2450 4666 \n",
"L 2450 531 \n",
"L 3481 531 \n",
"L 3481 0 \n",
"L 794 0 \n",
"L 794 531 \n",
"z\n",
"\" transform=\"scale(0.015625)\"/>\n",
" <path id=\"DejaVuSans-30\" d=\"M 2034 4250 \n",
"Q 1547 4250 1301 3770 \n",
"Q 1056 3291 1056 2328 \n",
"Q 1056 1369 1301 889 \n",
"Q 1547 409 2034 409 \n",
"Q 2525 409 2770 889 \n",
"Q 3016 1369 3016 2328 \n",
"Q 3016 3291 2770 3770 \n",
"Q 2525 4250 2034 4250 \n",
"z\n",
"M 2034 4750 \n",
"Q 2819 4750 3233 4129 \n",
"Q 3647 3509 3647 2328 \n",
"Q 3647 1150 3233 529 \n",
"Q 2819 -91 2034 -91 \n",
"Q 1250 -91 836 529 \n",
"Q 422 1150 422 2328 \n",
"Q 422 3509 836 4129 \n",
"Q 1250 4750 2034 4750 \n",
"z\n",
"\" transform=\"scale(0.015625)\"/>\n",
" </defs>\n",
" <use xlink:href=\"#DejaVuSans-31\"/>\n",
" <use xlink:href=\"#DejaVuSans-30\" x=\"63.623047\"/>\n",
" </g>\n",
" </g>\n",
" </g>\n",
" <g id=\"text_6\">\n",
" <!-- epoch -->\n",
" <g transform=\"translate(112.525 174.076563)scale(0.1 -0.1)\">\n",
" <defs>\n",
" <path id=\"DejaVuSans-65\" d=\"M 3597 1894 \n",
"L 3597 1613 \n",
"L 953 1613 \n",
"Q 991 1019 1311 708 \n",
"Q 1631 397 2203 397 \n",
"Q 2534 397 2845 478 \n",
"Q 3156 559 3463 722 \n",
"L 3463 178 \n",
"Q 3153 47 2828 -22 \n",
"Q 2503 -91 2169 -91 \n",
"Q 1331 -91 842 396 \n",
"Q 353 884 353 1716 \n",
"Q 353 2575 817 3079 \n",
"Q 1281 3584 2069 3584 \n",
"Q 2775 3584 3186 3129 \n",
"Q 3597 2675 3597 1894 \n",
"z\n",
"M 3022 2063 \n",
"Q 3016 2534 2758 2815 \n",
"Q 2500 3097 2075 3097 \n",
"Q 1594 3097 1305 2825 \n",
"Q 1016 2553 972 2059 \n",
"L 3022 2063 \n",
"z\n",
"\" transform=\"scale(0.015625)\"/>\n",
" <path id=\"DejaVuSans-70\" d=\"M 1159 525 \n",
"L 1159 -1331 \n",
"L 581 -1331 \n",
"L 581 3500 \n",
"L 1159 3500 \n",
"L 1159 2969 \n",
"Q 1341 3281 1617 3432 \n",
"Q 1894 3584 2278 3584 \n",
"Q 2916 3584 3314 3078 \n",
"Q 3713 2572 3713 1747 \n",
"Q 3713 922 3314 415 \n",
"Q 2916 -91 2278 -91 \n",
"Q 1894 -91 1617 61 \n",
"Q 1341 213 1159 525 \n",
"z\n",
"M 3116 1747 \n",
"Q 3116 2381 2855 2742 \n",
"Q 2594 3103 2138 3103 \n",
"Q 1681 3103 1420 2742 \n",
"Q 1159 2381 1159 1747 \n",
"Q 1159 1113 1420 752 \n",
"Q 1681 391 2138 391 \n",
"Q 2594 391 2855 752 \n",
"Q 3116 1113 3116 1747 \n",
"z\n",
"\" transform=\"scale(0.015625)\"/>\n",
" <path id=\"DejaVuSans-6f\" d=\"M 1959 3097 \n",
"Q 1497 3097 1228 2736 \n",
"Q 959 2375 959 1747 \n",
"Q 959 1119 1226 758 \n",
"Q 1494 397 1959 397 \n",
"Q 2419 397 2687 759 \n",
"Q 2956 1122 2956 1747 \n",
"Q 2956 2369 2687 2733 \n",
"Q 2419 3097 1959 3097 \n",
"z\n",
"M 1959 3584 \n",
"Q 2709 3584 3137 3096 \n",
"Q 3566 2609 3566 1747 \n",
"Q 3566 888 3137 398 \n",
"Q 2709 -91 1959 -91 \n",
"Q 1206 -91 779 398 \n",
"Q 353 888 353 1747 \n",
"Q 353 2609 779 3096 \n",
"Q 1206 3584 1959 3584 \n",
"z\n",
"\" transform=\"scale(0.015625)\"/>\n",
" <path id=\"DejaVuSans-63\" d=\"M 3122 3366 \n",
"L 3122 2828 \n",
"Q 2878 2963 2633 3030 \n",
"Q 2388 3097 2138 3097 \n",
"Q 1578 3097 1268 2742 \n",
"Q 959 2388 959 1747 \n",
"Q 959 1106 1268 751 \n",
"Q 1578 397 2138 397 \n",
"Q 2388 397 2633 464 \n",
"Q 2878 531 3122 666 \n",
"L 3122 134 \n",
"Q 2881 22 2623 -34 \n",
"Q 2366 -91 2075 -91 \n",
"Q 1284 -91 818 406 \n",
"Q 353 903 353 1747 \n",
"Q 353 2603 823 3093 \n",
"Q 1294 3584 2113 3584 \n",
"Q 2378 3584 2631 3529 \n",
"Q 2884 3475 3122 3366 \n",
"z\n",
"\" transform=\"scale(0.015625)\"/>\n",
" <path id=\"DejaVuSans-68\" d=\"M 3513 2113 \n",
"L 3513 0 \n",
"L 2938 0 \n",
"L 2938 2094 \n",
"Q 2938 2591 2744 2837 \n",
"Q 2550 3084 2163 3084 \n",
"Q 1697 3084 1428 2787 \n",
"Q 1159 2491 1159 1978 \n",
"L 1159 0 \n",
"L 581 0 \n",
"L 581 4863 \n",
"L 1159 4863 \n",
"L 1159 2956 \n",
"Q 1366 3272 1645 3428 \n",
"Q 1925 3584 2291 3584 \n",
"Q 2894 3584 3203 3211 \n",
"Q 3513 2838 3513 2113 \n",
"z\n",
"\" transform=\"scale(0.015625)\"/>\n",
" </defs>\n",
" <use xlink:href=\"#DejaVuSans-65\"/>\n",
" <use xlink:href=\"#DejaVuSans-70\" x=\"61.523438\"/>\n",
" <use xlink:href=\"#DejaVuSans-6f\" x=\"125\"/>\n",
" <use xlink:href=\"#DejaVuSans-63\" x=\"186.181641\"/>\n",
" <use xlink:href=\"#DejaVuSans-68\" x=\"241.162109\"/>\n",
" </g>\n",
" </g>\n",
" </g>\n",
" <g id=\"matplotlib.axis_2\">\n",
" <g id=\"ytick_1\">\n",
" <g id=\"line2d_11\">\n",
" <path d=\"M 30.103125 122.7 \n",
"L 225.403125 122.7 \n",
"\" clip-path=\"url(#p85ad12d352)\" style=\"fill: none; stroke: #b0b0b0; stroke-width: 0.8; stroke-linecap: square\"/>\n",
" </g>\n",
" <g id=\"line2d_12\">\n",
" <defs>\n",
" <path id=\"m4be5775045\" d=\"M 0 0 \n",
"L -3.5 0 \n",
"\" style=\"stroke: #000000; stroke-width: 0.8\"/>\n",
" </defs>\n",
" <g>\n",
" <use xlink:href=\"#m4be5775045\" x=\"30.103125\" y=\"122.7\" style=\"stroke: #000000; stroke-width: 0.8\"/>\n",
" </g>\n",
" </g>\n",
" <g id=\"text_7\">\n",
" <!-- 0.4 -->\n",
" <g transform=\"translate(7.2 126.499219)scale(0.1 -0.1)\">\n",
" <defs>\n",
" <path id=\"DejaVuSans-2e\" d=\"M 684 794 \n",
"L 1344 794 \n",
"L 1344 0 \n",
"L 684 0 \n",
"L 684 794 \n",
"z\n",
"\" transform=\"scale(0.015625)\"/>\n",
" </defs>\n",
" <use xlink:href=\"#DejaVuSans-30\"/>\n",
" <use xlink:href=\"#DejaVuSans-2e\" x=\"63.623047\"/>\n",
" <use xlink:href=\"#DejaVuSans-34\" x=\"95.410156\"/>\n",
" </g>\n",
" </g>\n",
" </g>\n",
" <g id=\"ytick_2\">\n",
" <g id=\"line2d_13\">\n",
" <path d=\"M 30.103125 76.5 \n",
"L 225.403125 76.5 \n",
"\" clip-path=\"url(#p85ad12d352)\" style=\"fill: none; stroke: #b0b0b0; stroke-width: 0.8; stroke-linecap: square\"/>\n",
" </g>\n",
" <g id=\"line2d_14\">\n",
" <g>\n",
" <use xlink:href=\"#m4be5775045\" x=\"30.103125\" y=\"76.5\" style=\"stroke: #000000; stroke-width: 0.8\"/>\n",
" </g>\n",
" </g>\n",
" <g id=\"text_8\">\n",
" <!-- 0.6 -->\n",
" <g transform=\"translate(7.2 80.299219)scale(0.1 -0.1)\">\n",
" <use xlink:href=\"#DejaVuSans-30\"/>\n",
" <use xlink:href=\"#DejaVuSans-2e\" x=\"63.623047\"/>\n",
" <use xlink:href=\"#DejaVuSans-36\" x=\"95.410156\"/>\n",
" </g>\n",
" </g>\n",
" </g>\n",
" <g id=\"ytick_3\">\n",
" <g id=\"line2d_15\">\n",
" <path d=\"M 30.103125 30.3 \n",
"L 225.403125 30.3 \n",
"\" clip-path=\"url(#p85ad12d352)\" style=\"fill: none; stroke: #b0b0b0; stroke-width: 0.8; stroke-linecap: square\"/>\n",
" </g>\n",
" <g id=\"line2d_16\">\n",
" <g>\n",
" <use xlink:href=\"#m4be5775045\" x=\"30.103125\" y=\"30.3\" style=\"stroke: #000000; stroke-width: 0.8\"/>\n",
" </g>\n",
" </g>\n",
" <g id=\"text_9\">\n",
" <!-- 0.8 -->\n",
" <g transform=\"translate(7.2 34.099219)scale(0.1 -0.1)\">\n",
" <use xlink:href=\"#DejaVuSans-30\"/>\n",
" <use xlink:href=\"#DejaVuSans-2e\" x=\"63.623047\"/>\n",
" <use xlink:href=\"#DejaVuSans-38\" x=\"95.410156\"/>\n",
" </g>\n",
" </g>\n",
" </g>\n",
" </g>\n",
" <g id=\"line2d_17\">\n",
" <path d=\"M 35.739269 -1 \n",
"L 51.803125 75.626533 \n",
"L 73.503125 95.460186 \n",
"L 95.203125 103.67903 \n",
"L 116.903125 110.265338 \n",
"L 138.603125 114.993629 \n",
"L 160.303125 118.355406 \n",
"L 182.003125 121.572363 \n",
"L 203.703125 124.377655 \n",
"L 225.403125 126.849041 \n",
"\" clip-path=\"url(#p85ad12d352)\" style=\"fill: none; stroke: #1f77b4; stroke-width: 1.5; stroke-linecap: square\"/>\n",
" </g>\n",
" <g id=\"line2d_18\">\n",
" <path d=\"M 30.103125 67.27925 \n",
"L 51.803125 33.04505 \n",
"L 73.503125 25.73005 \n",
"L 95.203125 23.49705 \n",
"L 116.903125 21.0138 \n",
"L 138.603125 19.289 \n",
"L 160.303125 17.9492 \n",
"L 182.003125 16.99055 \n",
"L 203.703125 16.0858 \n",
"L 225.403125 15.25805 \n",
"\" clip-path=\"url(#p85ad12d352)\" style=\"fill: none; stroke-dasharray: 5.55,2.4; stroke-dashoffset: 0; stroke: #bf00bf; stroke-width: 1.5\"/>\n",
" </g>\n",
" <g id=\"line2d_19\">\n",
" <path d=\"M 30.103125 40.3023 \n",
"L 51.803125 32.4714 \n",
"L 73.503125 27.5973 \n",
"L 95.203125 23.5086 \n",
"L 116.903125 23.8089 \n",
"L 138.603125 26.6502 \n",
"L 160.303125 24.756 \n",
"L 182.003125 22.446 \n",
"L 203.703125 18.1263 \n",
"L 225.403125 22.7925 \n",
"\" clip-path=\"url(#p85ad12d352)\" style=\"fill: none; stroke-dasharray: 9.6,2.4,1.5,2.4; stroke-dashoffset: 0; stroke: #008000; stroke-width: 1.5\"/>\n",
" </g>\n",
" <g id=\"patch_3\">\n",
" <path d=\"M 30.103125 145.8 \n",
"L 30.103125 7.2 \n",
"\" style=\"fill: none; stroke: #000000; stroke-width: 0.8; stroke-linejoin: miter; stroke-linecap: square\"/>\n",
" </g>\n",
" <g id=\"patch_4\">\n",
" <path d=\"M 225.403125 145.8 \n",
"L 225.403125 7.2 \n",
"\" style=\"fill: none; stroke: #000000; stroke-width: 0.8; stroke-linejoin: miter; stroke-linecap: square\"/>\n",
" </g>\n",
" <g id=\"patch_5\">\n",
" <path d=\"M 30.103125 145.8 \n",
"L 225.403125 145.8 \n",
"\" style=\"fill: none; stroke: #000000; stroke-width: 0.8; stroke-linejoin: miter; stroke-linecap: square\"/>\n",
" </g>\n",
" <g id=\"patch_6\">\n",
" <path d=\"M 30.103125 7.2 \n",
"L 225.403125 7.2 \n",
"\" style=\"fill: none; stroke: #000000; stroke-width: 0.8; stroke-linejoin: miter; stroke-linecap: square\"/>\n",
" </g>\n",
" <g id=\"legend_1\">\n",
" <g id=\"patch_7\">\n",
" <path d=\"M 140.634375 100.017188 \n",
"L 218.403125 100.017188 \n",
"Q 220.403125 100.017188 220.403125 98.017188 \n",
"L 220.403125 54.982812 \n",
"Q 220.403125 52.982812 218.403125 52.982812 \n",
"L 140.634375 52.982812 \n",
"Q 138.634375 52.982812 138.634375 54.982812 \n",
"L 138.634375 98.017188 \n",
"Q 138.634375 100.017188 140.634375 100.017188 \n",
"z\n",
"\" style=\"fill: #ffffff; opacity: 0.8; stroke: #cccccc; stroke-linejoin: miter\"/>\n",
" </g>\n",
" <g id=\"line2d_20\">\n",
" <path d=\"M 142.634375 61.08125 \n",
"L 152.634375 61.08125 \n",
"L 162.634375 61.08125 \n",
"\" style=\"fill: none; stroke: #1f77b4; stroke-width: 1.5; stroke-linecap: square\"/>\n",
" </g>\n",
" <g id=\"text_10\">\n",
" <!-- train loss -->\n",
" <g transform=\"translate(170.634375 64.58125)scale(0.1 -0.1)\">\n",
" <defs>\n",
" <path id=\"DejaVuSans-74\" d=\"M 1172 4494 \n",
"L 1172 3500 \n",
"L 2356 3500 \n",
"L 2356 3053 \n",
"L 1172 3053 \n",
"L 1172 1153 \n",
"Q 1172 725 1289 603 \n",
"Q 1406 481 1766 481 \n",
"L 2356 481 \n",
"L 2356 0 \n",
"L 1766 0 \n",
"Q 1100 0 847 248 \n",
"Q 594 497 594 1153 \n",
"L 594 3053 \n",
"L 172 3053 \n",
"L 172 3500 \n",
"L 594 3500 \n",
"L 594 4494 \n",
"L 1172 4494 \n",
"z\n",
"\" transform=\"scale(0.015625)\"/>\n",
" <path id=\"DejaVuSans-72\" d=\"M 2631 2963 \n",
"Q 2534 3019 2420 3045 \n",
"Q 2306 3072 2169 3072 \n",
"Q 1681 3072 1420 2755 \n",
"Q 1159 2438 1159 1844 \n",
"L 1159 0 \n",
"L 581 0 \n",
"L 581 3500 \n",
"L 1159 3500 \n",
"L 1159 2956 \n",
"Q 1341 3275 1631 3429 \n",
"Q 1922 3584 2338 3584 \n",
"Q 2397 3584 2469 3576 \n",
"Q 2541 3569 2628 3553 \n",
"L 2631 2963 \n",
"z\n",
"\" transform=\"scale(0.015625)\"/>\n",
" <path id=\"DejaVuSans-61\" d=\"M 2194 1759 \n",
"Q 1497 1759 1228 1600 \n",
"Q 959 1441 959 1056 \n",
"Q 959 750 1161 570 \n",
"Q 1363 391 1709 391 \n",
"Q 2188 391 2477 730 \n",
"Q 2766 1069 2766 1631 \n",
"L 2766 1759 \n",
"L 2194 1759 \n",
"z\n",
"M 3341 1997 \n",
"L 3341 0 \n",
"L 2766 0 \n",
"L 2766 531 \n",
"Q 2569 213 2275 61 \n",
"Q 1981 -91 1556 -91 \n",
"Q 1019 -91 701 211 \n",
"Q 384 513 384 1019 \n",
"Q 384 1609 779 1909 \n",
"Q 1175 2209 1959 2209 \n",
"L 2766 2209 \n",
"L 2766 2266 \n",
"Q 2766 2663 2505 2880 \n",
"Q 2244 3097 1772 3097 \n",
"Q 1472 3097 1187 3025 \n",
"Q 903 2953 641 2809 \n",
"L 641 3341 \n",
"Q 956 3463 1253 3523 \n",
"Q 1550 3584 1831 3584 \n",
"Q 2591 3584 2966 3190 \n",
"Q 3341 2797 3341 1997 \n",
"z\n",
"\" transform=\"scale(0.015625)\"/>\n",
" <path id=\"DejaVuSans-69\" d=\"M 603 3500 \n",
"L 1178 3500 \n",
"L 1178 0 \n",
"L 603 0 \n",
"L 603 3500 \n",
"z\n",
"M 603 4863 \n",
"L 1178 4863 \n",
"L 1178 4134 \n",
"L 603 4134 \n",
"L 603 4863 \n",
"z\n",
"\" transform=\"scale(0.015625)\"/>\n",
" <path id=\"DejaVuSans-6e\" d=\"M 3513 2113 \n",
"L 3513 0 \n",
"L 2938 0 \n",
"L 2938 2094 \n",
"Q 2938 2591 2744 2837 \n",
"Q 2550 3084 2163 3084 \n",
"Q 1697 3084 1428 2787 \n",
"Q 1159 2491 1159 1978 \n",
"L 1159 0 \n",
"L 581 0 \n",
"L 581 3500 \n",
"L 1159 3500 \n",
"L 1159 2956 \n",
"Q 1366 3272 1645 3428 \n",
"Q 1925 3584 2291 3584 \n",
"Q 2894 3584 3203 3211 \n",
"Q 3513 2838 3513 2113 \n",
"z\n",
"\" transform=\"scale(0.015625)\"/>\n",
" <path id=\"DejaVuSans-20\" transform=\"scale(0.015625)\"/>\n",
" <path id=\"DejaVuSans-6c\" d=\"M 603 4863 \n",
"L 1178 4863 \n",
"L 1178 0 \n",
"L 603 0 \n",
"L 603 4863 \n",
"z\n",
"\" transform=\"scale(0.015625)\"/>\n",
" <path id=\"DejaVuSans-73\" d=\"M 2834 3397 \n",
"L 2834 2853 \n",
"Q 2591 2978 2328 3040 \n",
"Q 2066 3103 1784 3103 \n",
"Q 1356 3103 1142 2972 \n",
"Q 928 2841 928 2578 \n",
"Q 928 2378 1081 2264 \n",
"Q 1234 2150 1697 2047 \n",
"L 1894 2003 \n",
"Q 2506 1872 2764 1633 \n",
"Q 3022 1394 3022 966 \n",
"Q 3022 478 2636 193 \n",
"Q 2250 -91 1575 -91 \n",
"Q 1294 -91 989 -36 \n",
"Q 684 19 347 128 \n",
"L 347 722 \n",
"Q 666 556 975 473 \n",
"Q 1284 391 1588 391 \n",
"Q 1994 391 2212 530 \n",
"Q 2431 669 2431 922 \n",
"Q 2431 1156 2273 1281 \n",
"Q 2116 1406 1581 1522 \n",
"L 1381 1569 \n",
"Q 847 1681 609 1914 \n",
"Q 372 2147 372 2553 \n",
"Q 372 3047 722 3315 \n",
"Q 1072 3584 1716 3584 \n",
"Q 2034 3584 2315 3537 \n",
"Q 2597 3491 2834 3397 \n",
"z\n",
"\" transform=\"scale(0.015625)\"/>\n",
" </defs>\n",
" <use xlink:href=\"#DejaVuSans-74\"/>\n",
" <use xlink:href=\"#DejaVuSans-72\" x=\"39.208984\"/>\n",
" <use xlink:href=\"#DejaVuSans-61\" x=\"80.322266\"/>\n",
" <use xlink:href=\"#DejaVuSans-69\" x=\"141.601562\"/>\n",
" <use xlink:href=\"#DejaVuSans-6e\" x=\"169.384766\"/>\n",
" <use xlink:href=\"#DejaVuSans-20\" x=\"232.763672\"/>\n",
" <use xlink:href=\"#DejaVuSans-6c\" x=\"264.550781\"/>\n",
" <use xlink:href=\"#DejaVuSans-6f\" x=\"292.333984\"/>\n",
" <use xlink:href=\"#DejaVuSans-73\" x=\"353.515625\"/>\n",
" <use xlink:href=\"#DejaVuSans-73\" x=\"405.615234\"/>\n",
" </g>\n",
" </g>\n",
" <g id=\"line2d_21\">\n",
" <path d=\"M 142.634375 75.759375 \n",
"L 152.634375 75.759375 \n",
"L 162.634375 75.759375 \n",
"\" style=\"fill: none; stroke-dasharray: 5.55,2.4; stroke-dashoffset: 0; stroke: #bf00bf; stroke-width: 1.5\"/>\n",
" </g>\n",
" <g id=\"text_11\">\n",
" <!-- train acc -->\n",
" <g transform=\"translate(170.634375 79.259375)scale(0.1 -0.1)\">\n",
" <use xlink:href=\"#DejaVuSans-74\"/>\n",
" <use xlink:href=\"#DejaVuSans-72\" x=\"39.208984\"/>\n",
" <use xlink:href=\"#DejaVuSans-61\" x=\"80.322266\"/>\n",
" <use xlink:href=\"#DejaVuSans-69\" x=\"141.601562\"/>\n",
" <use xlink:href=\"#DejaVuSans-6e\" x=\"169.384766\"/>\n",
" <use xlink:href=\"#DejaVuSans-20\" x=\"232.763672\"/>\n",
" <use xlink:href=\"#DejaVuSans-61\" x=\"264.550781\"/>\n",
" <use xlink:href=\"#DejaVuSans-63\" x=\"325.830078\"/>\n",
" <use xlink:href=\"#DejaVuSans-63\" x=\"380.810547\"/>\n",
" </g>\n",
" </g>\n",
" <g id=\"line2d_22\">\n",
" <path d=\"M 142.634375 90.4375 \n",
"L 152.634375 90.4375 \n",
"L 162.634375 90.4375 \n",
"\" style=\"fill: none; stroke-dasharray: 9.6,2.4,1.5,2.4; stroke-dashoffset: 0; stroke: #008000; stroke-width: 1.5\"/>\n",
" </g>\n",
" <g id=\"text_12\">\n",
" <!-- test acc -->\n",
" <g transform=\"translate(170.634375 93.9375)scale(0.1 -0.1)\">\n",
" <use xlink:href=\"#DejaVuSans-74\"/>\n",
" <use xlink:href=\"#DejaVuSans-65\" x=\"39.208984\"/>\n",
" <use xlink:href=\"#DejaVuSans-73\" x=\"100.732422\"/>\n",
" <use xlink:href=\"#DejaVuSans-74\" x=\"152.832031\"/>\n",
" <use xlink:href=\"#DejaVuSans-20\" x=\"192.041016\"/>\n",
" <use xlink:href=\"#DejaVuSans-61\" x=\"223.828125\"/>\n",
" <use xlink:href=\"#DejaVuSans-63\" x=\"285.107422\"/>\n",
" <use xlink:href=\"#DejaVuSans-63\" x=\"340.087891\"/>\n",
" </g>\n",
" </g>\n",
" </g>\n",
" </g>\n",
" </g>\n",
" <defs>\n",
" <clipPath id=\"p85ad12d352\">\n",
" <rect x=\"30.103125\" y=\"7.2\" width=\"195.3\" height=\"138.6\"/>\n",
" </clipPath>\n",
" </defs>\n",
"</svg>\n"
],
"text/plain": [
"<Figure size 350x250 with 1 Axes>"
]
},
"metadata": {},
"output_type": "display_data"
}
],
"source": [
"num_epochs, lr = 10, 0.1\n",
"updater = torch.optim.SGD(params, lr=lr)# 随机梯度下降\n",
"d2l.train_ch3(net, train_iter, test_iter, loss, num_epochs, updater)"
]
},
{
"cell_type": "markdown",
"id": "4da98919",
"metadata": {
"origin_pos": 30
},
"source": [
"为了对学习到的模型进行评估，我们将[**在一些测试数据上应用这个模型**]。\n"
]
},
{
"cell_type": "code",
"execution_count": 8,
"id": "8230ba7c",
"metadata": {
"execution": {
"iopub.execute_input": "2023-08-18T07:00:19.755336Z",
"iopub.status.busy": "2023-08-18T07:00:19.754506Z",
"iopub.status.idle": "2023-08-18T07:00:20.323813Z",
"shell.execute_reply": "2023-08-18T07:00:20.322738Z"
},
"origin_pos": 31,
"tab": [
"pytorch"
]
},
"outputs": [
{
"data": {
"image/svg+xml": [
"<?xml version=\"1.0\" encoding=\"utf-8\" standalone=\"no\"?>\n",
"<!DOCTYPE svg PUBLIC \"-//W3C//DTD SVG 1.1//EN\"\n",
" \"http://www.w3.org/Graphics/SVG/1.1/DTD/svg11.dtd\">\n",
"<svg xmlns:xlink=\"http://www.w3.org/1999/xlink\" width=\"520.1pt\" height=\"118.198357pt\" viewBox=\"0 0 520.1 118.198357\" xmlns=\"http://www.w3.org/2000/svg\" version=\"1.1\">\n",
" <metadata>\n",
" <rdf:RDF xmlns:dc=\"http://purl.org/dc/elements/1.1/\" xmlns:cc=\"http://creativecommons.org/ns#\" xmlns:rdf=\"http://www.w3.org/1999/02/22-rdf-syntax-ns#\">\n",
" <cc:Work>\n",
" <dc:type rdf:resource=\"http://purl.org/dc/dcmitype/StillImage\"/>\n",
" <dc:date>2023-10-31T20:43:43.106424</dc:date>\n",
" <dc:format>image/svg+xml</dc:format>\n",
" <dc:creator>\n",
" <cc:Agent>\n",
" <dc:title>Matplotlib v3.5.1, https://matplotlib.org/</dc:title>\n",
" </cc:Agent>\n",
" </dc:creator>\n",
" </cc:Work>\n",
" </rdf:RDF>\n",
" </metadata>\n",
" <defs>\n",
" <style type=\"text/css\">*{stroke-linejoin: round; stroke-linecap: butt}</style>\n",
" </defs>\n",
" <g id=\"figure_1\">\n",
" <g id=\"patch_1\">\n",
" <path d=\"M 0 118.198357 \n",
"L 520.1 118.198357 \n",
"L 520.1 0 \n",
"L 0 0 \n",
"z\n",
"\" style=\"fill: #ffffff\"/>\n",
" </g>\n",
" <g id=\"axes_1\">\n",
" <g id=\"patch_2\">\n",
" <path d=\"M 10.7 107.498357 \n",
"L 82.442857 107.498357 \n",
"L 82.442857 35.7555 \n",
"L 10.7 35.7555 \n",
"z\n",
"\" style=\"fill: #ffffff\"/>\n",
" </g>\n",
" <g clip-path=\"url(#p8a44a1479b)\">\n",
" <image xlink:href=\"data:image/png;base64,\n",
"iVBORw0KGgoAAAANSUhEUgAAAGQAAABkCAYAAABw4pVUAAAGQUlEQVR4nO2cS28cRRDHe2Z29mWvH2uv7SR+xHbiyFzggAiJxAEFxAWJCwef4B4OOSBx4AoSQSIXDnwChIDwAQAFcUAyDwkhLMUosYljEDg2tndtr2cf8+I21f82s7EtWZSl+p2qXb2zM/N3V1X39Kz1gvVqrAQ22P/3CQiICMIMEYQZIggzRBBmiCDMEEGYIYIwQwRhhgjCDBGEGSIIM0QQZoggzBBBmCGCMEMEYYYIwgwRhBkiCDMyR+ns9Pcn9u61GfBVZxw66DNV8E2XN6E93kX+czns6yjacxEqC3x+RKe7WD8Dvu8fTEK7/9t8Ylc+XQBftL+vDkP0zRi0n6/ch/bC3rnEfrTfA76t/WJiB4EDPr9N1zHzxgPwyQhhhgjCDBGEGdZLI9dho9z9t6YSu/LEP9B5ogfjvU4zcBN73esG32a1BO2glk1sdxfja+TS6cSYQlRc9hP7qak/wFfJ16E9VaC8Fcb4f/f24L3Efn/rIvi+Xp9N7I9nPgFf2clBO4zT9xh6MZ3rV944+Jabw4n93Y1nwScjhBkiCDNEEGZYl+c+gEAYvLaV2FtLA9A5v076OW08kB6mbdOHaULp04vImAlFrtYtQp9f0k7VyC9h3ojnA63EzGRDcHUXm4k9O7Ch0uhxm9DOWGFKT6UeNXEeMpSjnLbdLoLPCyiHtm7gPZYRwgwRhBmZ0mc/wB/q0eXE7j6LetkB2c1BPFDsUMiwQivVpxSWs0EX+qK8FqeMsKSy5HNqZqzDpqWdehzhgbY3KLzMV7FEtzconBSmd1Un6lUtFLXxXmV6KG7nCxjDnxul5ZLFs1gSywhhhgjCDBGEGQeW37tv/5jYfecxvjVmhhLbX8Natt1F2gYFzAt75/E7Qs1v+xjfs+t0SsaKByyrmGWuWSKHdTpO5BsH0nJaoa8BLq+HrqvVwtuTzaaXvW4flsiVPip7d7wC+GptakcuXr+MEGaIIMywXnTnYOzHQZDW99hkxkah3bhEq53bs7iC2hjRwlkbh7NDk2/lG+VyUDJKa33VOI+hxi3SSmxXsQW+nEvXv73TBb4oMqYBNsVJ33PBp7QwaXf74OopUZis3MTrlxHCDBGEGSIIM6yOP61hm8u0GnGHtYoovTx8HM4F2j3ycA53lujlcraK+SXAcK+iLPVtl/F8smUqUf1NLEkzZSxfdSwbb1UmQ8f1tnFF19JKZDePeXl8kJ682tf+BJ+MEGaIIMzovFHuKKEn1voaoc5ysB0HWhlobBQIl1cSe+zdFfCtvXk1sfcuYBjIr+GlOE0r1ZcZoc+2Lfx+v0Hlq+3i9ecKWL7ms9T2jBXtnFZal4oYBld/omnApJKQxRoRhBkiCDOOtNkasMxdbFoMNXJPfIRcZGW01V5jGefMrfnEdq5fBV/tSYzvfQuUC8yNFF6DngraJfxcXCWfNWBsjijgMovraH5j8mBZ6bOJiS/TS2sZIcwQQZghgjDj+Dmkw0bjjvlFKZynGEswet7Q84npG/poHnzeO1eg3Rim79TnJEopFexSnhgcrYGvps0nbAfPzTdevAkjc1sMEWm+PS8PvsFfadeJmV1lhDBDBGHG8UNWJzqFM6UOvSRz4OmlHuqMY0zfWoT2bzcvJbbVMp70NajttfBJ32iFVmLXd/C9Fv0JoVJKlYv05K+WxeXmwKdbG7aMUFfbUWnICGGGCMIMEYQZJ5NDTgotb1huFlxmXJ78gvquvoz/d06LStJGHXd9xCV6hz0yytrYeOlx+R490XT7cVmlV9tZsvl3rzosMkKYIYIw43SFLA146vgfuHd+TuzeiziLrz5Nn83/jrPo3OsPE3viMedQ0ezMBP4Mx1+vULuIr6B0REYIM0QQZoggzDi1OeQA5qY+rUQeubOOXQPa7D10+y74jrvFL1jF3SPDH1J75b0rZvdUZIQwQwRhxukNWeaKcpwebMIl/NW2gaX0B0SwySI0vB1Wse0i7u2NPC+xpz7H16s7rYXLCGGGCMIMEYQZpzeHnBDHfcdSzxkHjvnL3VSfiYwQZoggzBBBmCGCMEMEYYYIwgwRhBkiCDNEEGaIIMwQQZghgjBDBGGGCMIMEYQZIggzRBBmiCDMEEGYIYIwQwRhhgjCDBGEGSIIM/4F94O8Ngd0y+4AAAAASUVORK5CYII=\" id=\"imagec016e18c7a\" transform=\"scale(1 -1)translate(0 -72)\" x=\"10.7\" y=\"-35.498357\" width=\"72\" height=\"72\"/>\n",
" </g>\n",
" <g id=\"patch_3\">\n",
" <path d=\"M 10.7 107.498357 \n",
"L 10.7 35.7555 \n",
"\" style=\"fill: none; stroke: #000000; stroke-width: 0.8; stroke-linejoin: miter; stroke-linecap: square\"/>\n",
" </g>\n",
" <g id=\"patch_4\">\n",
" <path d=\"M 82.442857 107.498357 \n",
"L 82.442857 35.7555 \n",
"\" style=\"fill: none; stroke: #000000; stroke-width: 0.8; stroke-linejoin: miter; stroke-linecap: square\"/>\n",
" </g>\n",
" <g id=\"patch_5\">\n",
" <path d=\"M 10.7 107.498357 \n",
"L 82.442857 107.498357 \n",
"\" style=\"fill: none; stroke: #000000; stroke-width: 0.8; stroke-linejoin: miter; stroke-linecap: square\"/>\n",
" </g>\n",
" <g id=\"patch_6\">\n",
" <path d=\"M 10.7 35.7555 \n",
"L 82.442857 35.7555 \n",
"\" style=\"fill: none; stroke: #000000; stroke-width: 0.8; stroke-linejoin: miter; stroke-linecap: square\"/>\n",
" </g>\n",
" <g id=\"text_1\">\n",
" <!-- ankle boot -->\n",
" <g transform=\"translate(14.848304 16.318125)scale(0.12 -0.12)\">\n",
" <defs>\n",
" <path id=\"DejaVuSans-61\" d=\"M 2194 1759 \n",
"Q 1497 1759 1228 1600 \n",
"Q 959 1441 959 1056 \n",
"Q 959 750 1161 570 \n",
"Q 1363 391 1709 391 \n",
"Q 2188 391 2477 730 \n",
"Q 2766 1069 2766 1631 \n",
"L 2766 1759 \n",
"L 2194 1759 \n",
"z\n",
"M 3341 1997 \n",
"L 3341 0 \n",
"L 2766 0 \n",
"L 2766 531 \n",
"Q 2569 213 2275 61 \n",
"Q 1981 -91 1556 -91 \n",
"Q 1019 -91 701 211 \n",
"Q 384 513 384 1019 \n",
"Q 384 1609 779 1909 \n",
"Q 1175 2209 1959 2209 \n",
"L 2766 2209 \n",
"L 2766 2266 \n",
"Q 2766 2663 2505 2880 \n",
"Q 2244 3097 1772 3097 \n",
"Q 1472 3097 1187 3025 \n",
"Q 903 2953 641 2809 \n",
"L 641 3341 \n",
"Q 956 3463 1253 3523 \n",
"Q 1550 3584 1831 3584 \n",
"Q 2591 3584 2966 3190 \n",
"Q 3341 2797 3341 1997 \n",
"z\n",
"\" transform=\"scale(0.015625)\"/>\n",
" <path id=\"DejaVuSans-6e\" d=\"M 3513 2113 \n",
"L 3513 0 \n",
"L 2938 0 \n",
"L 2938 2094 \n",
"Q 2938 2591 2744 2837 \n",
"Q 2550 3084 2163 3084 \n",
"Q 1697 3084 1428 2787 \n",
"Q 1159 2491 1159 1978 \n",
"L 1159 0 \n",
"L 581 0 \n",
"L 581 3500 \n",
"L 1159 3500 \n",
"L 1159 2956 \n",
"Q 1366 3272 1645 3428 \n",
"Q 1925 3584 2291 3584 \n",
"Q 2894 3584 3203 3211 \n",
"Q 3513 2838 3513 2113 \n",
"z\n",
"\" transform=\"scale(0.015625)\"/>\n",
" <path id=\"DejaVuSans-6b\" d=\"M 581 4863 \n",
"L 1159 4863 \n",
"L 1159 1991 \n",
"L 2875 3500 \n",
"L 3609 3500 \n",
"L 1753 1863 \n",
"L 3688 0 \n",
"L 2938 0 \n",
"L 1159 1709 \n",
"L 1159 0 \n",
"L 581 0 \n",
"L 581 4863 \n",
"z\n",
"\" transform=\"scale(0.015625)\"/>\n",
" <path id=\"DejaVuSans-6c\" d=\"M 603 4863 \n",
"L 1178 4863 \n",
"L 1178 0 \n",
"L 603 0 \n",
"L 603 4863 \n",
"z\n",
"\" transform=\"scale(0.015625)\"/>\n",
" <path id=\"DejaVuSans-65\" d=\"M 3597 1894 \n",
"L 3597 1613 \n",
"L 953 1613 \n",
"Q 991 1019 1311 708 \n",
"Q 1631 397 2203 397 \n",
"Q 2534 397 2845 478 \n",
"Q 3156 559 3463 722 \n",
"L 3463 178 \n",
"Q 3153 47 2828 -22 \n",
"Q 2503 -91 2169 -91 \n",
"Q 1331 -91 842 396 \n",
"Q 353 884 353 1716 \n",
"Q 353 2575 817 3079 \n",
"Q 1281 3584 2069 3584 \n",
"Q 2775 3584 3186 3129 \n",
"Q 3597 2675 3597 1894 \n",
"z\n",
"M 3022 2063 \n",
"Q 3016 2534 2758 2815 \n",
"Q 2500 3097 2075 3097 \n",
"Q 1594 3097 1305 2825 \n",
"Q 1016 2553 972 2059 \n",
"L 3022 2063 \n",
"z\n",
"\" transform=\"scale(0.015625)\"/>\n",
" <path id=\"DejaVuSans-20\" transform=\"scale(0.015625)\"/>\n",
" <path id=\"DejaVuSans-62\" d=\"M 3116 1747 \n",
"Q 3116 2381 2855 2742 \n",
"Q 2594 3103 2138 3103 \n",
"Q 1681 3103 1420 2742 \n",
"Q 1159 2381 1159 1747 \n",
"Q 1159 1113 1420 752 \n",
"Q 1681 391 2138 391 \n",
"Q 2594 391 2855 752 \n",
"Q 3116 1113 3116 1747 \n",
"z\n",
"M 1159 2969 \n",
"Q 1341 3281 1617 3432 \n",
"Q 1894 3584 2278 3584 \n",
"Q 2916 3584 3314 3078 \n",
"Q 3713 2572 3713 1747 \n",
"Q 3713 922 3314 415 \n",
"Q 2916 -91 2278 -91 \n",
"Q 1894 -91 1617 61 \n",
"Q 1341 213 1159 525 \n",
"L 1159 0 \n",
"L 581 0 \n",
"L 581 4863 \n",
"L 1159 4863 \n",
"L 1159 2969 \n",
"z\n",
"\" transform=\"scale(0.015625)\"/>\n",
" <path id=\"DejaVuSans-6f\" d=\"M 1959 3097 \n",
"Q 1497 3097 1228 2736 \n",
"Q 959 2375 959 1747 \n",
"Q 959 1119 1226 758 \n",
"Q 1494 397 1959 397 \n",
"Q 2419 397 2687 759 \n",
"Q 2956 1122 2956 1747 \n",
"Q 2956 2369 2687 2733 \n",
"Q 2419 3097 1959 3097 \n",
"z\n",
"M 1959 3584 \n",
"Q 2709 3584 3137 3096 \n",
"Q 3566 2609 3566 1747 \n",
"Q 3566 888 3137 398 \n",
"Q 2709 -91 1959 -91 \n",
"Q 1206 -91 779 398 \n",
"Q 353 888 353 1747 \n",
"Q 353 2609 779 3096 \n",
"Q 1206 3584 1959 3584 \n",
"z\n",
"\" transform=\"scale(0.015625)\"/>\n",
" <path id=\"DejaVuSans-74\" d=\"M 1172 4494 \n",
"L 1172 3500 \n",
"L 2356 3500 \n",
"L 2356 3053 \n",
"L 1172 3053 \n",
"L 1172 1153 \n",
"Q 1172 725 1289 603 \n",
"Q 1406 481 1766 481 \n",
"L 2356 481 \n",
"L 2356 0 \n",
"L 1766 0 \n",
"Q 1100 0 847 248 \n",
"Q 594 497 594 1153 \n",
"L 594 3053 \n",
"L 172 3053 \n",
"L 172 3500 \n",
"L 594 3500 \n",
"L 594 4494 \n",
"L 1172 4494 \n",
"z\n",
"\" transform=\"scale(0.015625)\"/>\n",
" </defs>\n",
" <use xlink:href=\"#DejaVuSans-61\"/>\n",
" <use xlink:href=\"#DejaVuSans-6e\" x=\"61.279297\"/>\n",
" <use xlink:href=\"#DejaVuSans-6b\" x=\"124.658203\"/>\n",
" <use xlink:href=\"#DejaVuSans-6c\" x=\"182.568359\"/>\n",
" <use xlink:href=\"#DejaVuSans-65\" x=\"210.351562\"/>\n",
" <use xlink:href=\"#DejaVuSans-20\" x=\"271.875\"/>\n",
" <use xlink:href=\"#DejaVuSans-62\" x=\"303.662109\"/>\n",
" <use xlink:href=\"#DejaVuSans-6f\" x=\"367.138672\"/>\n",
" <use xlink:href=\"#DejaVuSans-6f\" x=\"428.320312\"/>\n",
" <use xlink:href=\"#DejaVuSans-74\" x=\"489.501953\"/>\n",
" </g>\n",
" <!-- ankle boot -->\n",
" <g transform=\"translate(14.848304 29.7555)scale(0.12 -0.12)\">\n",
" <use xlink:href=\"#DejaVuSans-61\"/>\n",
" <use xlink:href=\"#DejaVuSans-6e\" x=\"61.279297\"/>\n",
" <use xlink:href=\"#DejaVuSans-6b\" x=\"124.658203\"/>\n",
" <use xlink:href=\"#DejaVuSans-6c\" x=\"182.568359\"/>\n",
" <use xlink:href=\"#DejaVuSans-65\" x=\"210.351562\"/>\n",
" <use xlink:href=\"#DejaVuSans-20\" x=\"271.875\"/>\n",
" <use xlink:href=\"#DejaVuSans-62\" x=\"303.662109\"/>\n",
" <use xlink:href=\"#DejaVuSans-6f\" x=\"367.138672\"/>\n",
" <use xlink:href=\"#DejaVuSans-6f\" x=\"428.320312\"/>\n",
" <use xlink:href=\"#DejaVuSans-74\" x=\"489.501953\"/>\n",
" </g>\n",
" </g>\n",
" </g>\n",
" <g id=\"axes_2\">\n",
" <g id=\"patch_7\">\n",
" <path d=\"M 96.791429 107.498357 \n",
"L 168.534286 107.498357 \n",
"L 168.534286 35.7555 \n",
"L 96.791429 35.7555 \n",
"z\n",
"\" style=\"fill: #ffffff\"/>\n",
" </g>\n",
" <g clip-path=\"url(#p9f0c7898d2)\">\n",
" <image xlink:href=\"data:image/png;base64,\n",
"iVBORw0KGgoAAAANSUhEUgAAAGQAAABkCAYAAABw4pVUAAAJCUlEQVR4nO2dXWwc1RXH78zsetfrz9gB59vESlIkSIggifkqIg2FBCpoKxQZhET4euLrARWkSjwgIbV9KfACLanUEglVVFGRQFDCZzBtAypUgSQOTjBgO4nt2AnrrLPe3dmZ4QXm3v8xd2xfDLqyzu/pHp2ZO2OfueeeOXPuXeca5+ZIGBBcfTHIVzz1Qdw+VakH3ec/z+K5+fG47aRrQBdVfSk4rsmtfS8c11HupYq6VArk4ivL4/ay+jzoBp5YE7frdn8gZsqP/xczibBBLIMNYhmp6Q/5bgrLMyA/ds6huD0eToLu1qbteLIyh0R+RX+RKDC9PWOiMEG5/nwQX75gZ9xucmtBt3qTPLZj98yvzyPEMtgglmHsskY3YLQ8VJ2I2yUaSLvzw+5OBcPg04F0qYVwAnTVejN3Oz/+U/MINohlsEEsw3gO2bThCMiFSKYclnge6ta1gXznv/bF7R2NJ0H3fKE1bqcd9NlJeEKfAfIcjGUrkby/Yojhu3o/a969HXSrHi+BvDItU0T7y2XQdV70Wdz+SntnU+ERYhlsEMswdlmnSnUgZx3pMupdzO66FXQnG7P9cbvPx/BwfQbf8nUEwgE5jFB2Hb0LyzrymqUI3WsQyezz0oV50JUXLdT2mXHw79h/fGncbhentedReIRYBhvEMtgglmE8h9SmfJDbvIzmSCHKD6APbfNkGNrr50CXdvQphzDSPz90TkkL2Q89r+R4Wl1fVc5h97W/A7qHf3Gr9vpZElpXfbN/LY8Qy2CDWIaxy6pP4ZtpEClhJnoPcSqPRQ8jgXwO6Bt2jZBDv0KeF9Wd0fOoy1JDW5/0ox7rk/PWpGU43/XxNtAleFPhkb85mPS++8Bp4BFiGWwQy2CDWIbxHJLxMBM7FspihRUuFr+5RzG0zXfKEPkcD1Mlo4EsFsjOIttL5xAq6yhF6Sk9fUt+sBk0bReMglyOZOg/5cmumj3rPEIsgw1iGWwQyzBPnXiYOilFep/t+qj7/+TKuH1HUy/oBqvSp89mDkn6YkhJK2mOpDmk/SV88bjs90fxSOXdi751pPP8HjIvYINYhrHLakmfBTmb8IWOJmlfGLwkbt+/oB90fmR8S6Qf/bOWUdxS1vG1x+UOngD5qgZ0rxNK2Jtz0UV5xZmF3RQeIZbBBrEMNohlGDvsyQDDxayj95mVVZgeOT60QArr8NizoZpWKYBupukQCj1PTcfXOXR9ipwLKh1Y4Lc1h58cDleU4kDynyRT7IzhEWIZbBDLMHZZ5RBdVpJlN3ZgaPvxnvM1R2Idrk+K2FxFZ+q+hMDiuBYP63X9SBb5FRfrCzeEEKIcqcUS+FZfO2q02pxHiG2wQSyDDWIZxnPI5xOtIC8gy4JVlmbzIA9+ql97nHNkaEnnCfXpma7qRIUWwyVVrxz2ZTokIYE95Zo+6ad2dOaZahUeIZbBBrEMY5d1bLwJZE/ZuUddIi2EEBvrvwB574JObb+NbkmrU93LbFyWS+puS0pG2RX4pp4PZdh7cqO2yyn3cK6H62WckMPeeQEbxDLYIJZhPIdMjOvD3P4q6rbV4Ze3XV0ylTJA5pt8KMNpWoDQ7Bbjds7FzCtd3qzSQOalvHLsaVJIMVyVc+OOa/dq+5yO7LBM9yZtMEThEWIZbBDLYINYhvEc4o7VTH/QN9Dd1p5fJbdY6/VRh1/lcJ444ku/rBZlCyFER/qM9vpfVnHB0JVZ2U+OFIZfkpG73QV1uCnGQLUIctrRF8NFh45qdUnwCLEMNohlmNf2juhtSZc2+ySLcFbZaXJJCgsgOv75UNyOUnjiPZe/G7d/uxCL1n4z/FOQz8uOxe17mwdBt3dSpkf+sPZSvLcXZWFD99oXQZdUv0yhmzDPFB4hlsEGsQw2iGUYzyHZhKqKNEl3l8nuxEXFF6vrwoUQYvV9+o3ruzfIfPiuhzeBrn37AZA/ETIF4/Xg9Z/5801xe1kLVsScOKgUx63F69OtnNJKUmQi1H82mA08QiyDDWIZxi6rYVC/Zzvd3U2Qr3vqJpnddKQr6yz2HPsIVDd0Lovb7duPge6OXnQ9XQ3yLZtuUPmPw/Le//LfF0D3u5Ob4zbNRHtk4VqzK13WnuIiMRfwCLEMNohlsEEswzzsHZqY/qBvKISYRlicktnX18+Q2DLU74E0slX+5lPrTpxD/tZ1Pci7ziqTU4qEqz0fxu27r74NdNG4zBqvOIBZYjoX5ZTH+b3CGoFw6mRewAaxDGOXFfX1a3X0TT0fot0XK+19YytBlxID2n4/fOyZuH3dzvWgO7KjAe+h0Bi3V+zB2Lr0a1moVzuEOrcJMwcz5a0BdFlLRI9RPzxCLIMNYhlsEMswnkPCkj67SecQWpCgkrkFvxgm/XLTJxV5zbZ9jai87H3teTf2nAJZ/YJ4/dqfge7oIz/R9kN3Sc0qBeb+gSZ6uBE8QiyDDWIZbBDLmJu9kIQQXwWyiGzqYhq93Xv/uAzkjmelXI7+B7rhqnzX2NXeDbov+jGV80ZRvhdcUdsHuv+UZLH1qwfeBt3rxffitvp7WEIIsbpmGGRfWZu+5N/6bZ5mA48Qy2CDWMacuaxeX785Mt0io7skh3rflr9iR1tk8xD5qdOsK93Ca0VcD7KcFNz9sl7W1u4vN4OuRinke62Ia1BaPOn6Wj3c0icfYvg+Esg64NqeIdCZ5Xp5hFgHG8Qy2CCW4Vzj3Gy2oJpw/JHL4/bBB58GHa0sKSlbO4XkmXCV4jO6Zn3qpseSKZvxK9tp1JH1iHgeXl9dq0ivty2H60Vu/PRXUtiCXzBN4RFiGWwQy5gzl6Vy5E9Yd/vq1idB7khLVzASoDsZDqTLKIT4E66LlJC0ycW8cA3ZhDOvJJxPVMnXRGUpdAtxi4uUCD3j4FvBg8c3gzzQabjTZQI8QiyDDWIZbBDL+EHmEIpbh5UcfY/K3ZPvuuFN0F1YK7/mdWbwS9+JQDr4dTU4vyQxFqCvV39Hka75eKC3K26Hz50Lusa/679KzhU8QiyDDWIZP4rLmg1ea0vcPrN5NejqduuXuyURbL4Y5NQZGWpHHx0y6vOHgkeIZbBBLIMNYhlfA778lLHwiudmAAAAAElFTkSuQmCC\" id=\"image2dd73e881f\" transform=\"scale(1 -1)translate(0 -72)\" x=\"96.791429\" y=\"-35.498357\" width=\"72\" height=\"72\"/>\n",
" </g>\n",
" <g id=\"patch_8\">\n",
" <path d=\"M 96.791429 107.498357 \n",
"L 96.791429 35.7555 \n",
"\" style=\"fill: none; stroke: #000000; stroke-width: 0.8; stroke-linejoin: miter; stroke-linecap: square\"/>\n",
" </g>\n",
" <g id=\"patch_9\">\n",
" <path d=\"M 168.534286 107.498357 \n",
"L 168.534286 35.7555 \n",
"\" style=\"fill: none; stroke: #000000; stroke-width: 0.8; stroke-linejoin: miter; stroke-linecap: square\"/>\n",
" </g>\n",
" <g id=\"patch_10\">\n",
" <path d=\"M 96.791429 107.498357 \n",
"L 168.534286 107.498357 \n",
"\" style=\"fill: none; stroke: #000000; stroke-width: 0.8; stroke-linejoin: miter; stroke-linecap: square\"/>\n",
" </g>\n",
" <g id=\"patch_11\">\n",
" <path d=\"M 96.791429 35.7555 \n",
"L 168.534286 35.7555 \n",
"\" style=\"fill: none; stroke: #000000; stroke-width: 0.8; stroke-linejoin: miter; stroke-linecap: square\"/>\n",
" </g>\n",
" <g id=\"text_2\">\n",
" <!-- pullover -->\n",
" <g transform=\"translate(108.336607 16.318125)scale(0.12 -0.12)\">\n",
" <defs>\n",
" <path id=\"DejaVuSans-70\" d=\"M 1159 525 \n",
"L 1159 -1331 \n",
"L 581 -1331 \n",
"L 581 3500 \n",
"L 1159 3500 \n",
"L 1159 2969 \n",
"Q 1341 3281 1617 3432 \n",
"Q 1894 3584 2278 3584 \n",
"Q 2916 3584 3314 3078 \n",
"Q 3713 2572 3713 1747 \n",
"Q 3713 922 3314 415 \n",
"Q 2916 -91 2278 -91 \n",
"Q 1894 -91 1617 61 \n",
"Q 1341 213 1159 525 \n",
"z\n",
"M 3116 1747 \n",
"Q 3116 2381 2855 2742 \n",
"Q 2594 3103 2138 3103 \n",
"Q 1681 3103 1420 2742 \n",
"Q 1159 2381 1159 1747 \n",
"Q 1159 1113 1420 752 \n",
"Q 1681 391 2138 391 \n",
"Q 2594 391 2855 752 \n",
"Q 3116 1113 3116 1747 \n",
"z\n",
"\" transform=\"scale(0.015625)\"/>\n",
" <path id=\"DejaVuSans-75\" d=\"M 544 1381 \n",
"L 544 3500 \n",
"L 1119 3500 \n",
"L 1119 1403 \n",
"Q 1119 906 1312 657 \n",
"Q 1506 409 1894 409 \n",
"Q 2359 409 2629 706 \n",
"Q 2900 1003 2900 1516 \n",
"L 2900 3500 \n",
"L 3475 3500 \n",
"L 3475 0 \n",
"L 2900 0 \n",
"L 2900 538 \n",
"Q 2691 219 2414 64 \n",
"Q 2138 -91 1772 -91 \n",
"Q 1169 -91 856 284 \n",
"Q 544 659 544 1381 \n",
"z\n",
"M 1991 3584 \n",
"L 1991 3584 \n",
"z\n",
"\" transform=\"scale(0.015625)\"/>\n",
" <path id=\"DejaVuSans-76\" d=\"M 191 3500 \n",
"L 800 3500 \n",
"L 1894 563 \n",
"L 2988 3500 \n",
"L 3597 3500 \n",
"L 2284 0 \n",
"L 1503 0 \n",
"L 191 3500 \n",
"z\n",
"\" transform=\"scale(0.015625)\"/>\n",
" <path id=\"DejaVuSans-72\" d=\"M 2631 2963 \n",
"Q 2534 3019 2420 3045 \n",
"Q 2306 3072 2169 3072 \n",
"Q 1681 3072 1420 2755 \n",
"Q 1159 2438 1159 1844 \n",
"L 1159 0 \n",
"L 581 0 \n",
"L 581 3500 \n",
"L 1159 3500 \n",
"L 1159 2956 \n",
"Q 1341 3275 1631 3429 \n",
"Q 1922 3584 2338 3584 \n",
"Q 2397 3584 2469 3576 \n",
"Q 2541 3569 2628 3553 \n",
"L 2631 2963 \n",
"z\n",
"\" transform=\"scale(0.015625)\"/>\n",
" </defs>\n",
" <use xlink:href=\"#DejaVuSans-70\"/>\n",
" <use xlink:href=\"#DejaVuSans-75\" x=\"63.476562\"/>\n",
" <use xlink:href=\"#DejaVuSans-6c\" x=\"126.855469\"/>\n",
" <use xlink:href=\"#DejaVuSans-6c\" x=\"154.638672\"/>\n",
" <use xlink:href=\"#DejaVuSans-6f\" x=\"182.421875\"/>\n",
" <use xlink:href=\"#DejaVuSans-76\" x=\"243.603516\"/>\n",
" <use xlink:href=\"#DejaVuSans-65\" x=\"302.783203\"/>\n",
" <use xlink:href=\"#DejaVuSans-72\" x=\"364.306641\"/>\n",
" </g>\n",
" <!-- pullover -->\n",
" <g transform=\"translate(108.336607 29.7555)scale(0.12 -0.12)\">\n",
" <use xlink:href=\"#DejaVuSans-70\"/>\n",
" <use xlink:href=\"#DejaVuSans-75\" x=\"63.476562\"/>\n",
" <use xlink:href=\"#DejaVuSans-6c\" x=\"126.855469\"/>\n",
" <use xlink:href=\"#DejaVuSans-6c\" x=\"154.638672\"/>\n",
" <use xlink:href=\"#DejaVuSans-6f\" x=\"182.421875\"/>\n",
" <use xlink:href=\"#DejaVuSans-76\" x=\"243.603516\"/>\n",
" <use xlink:href=\"#DejaVuSans-65\" x=\"302.783203\"/>\n",
" <use xlink:href=\"#DejaVuSans-72\" x=\"364.306641\"/>\n",
" </g>\n",
" </g>\n",
" </g>\n",
" <g id=\"axes_3\">\n",
" <g id=\"patch_12\">\n",
" <path d=\"M 182.882857 107.498357 \n",
"L 254.625714 107.498357 \n",
"L 254.625714 35.7555 \n",
"L 182.882857 35.7555 \n",
"z\n",
"\" style=\"fill: #ffffff\"/>\n",
" </g>\n",
" <g clip-path=\"url(#pfd11250cbb)\">\n",
" <image xlink:href=\"data:image/png;base64,\n",
"iVBORw0KGgoAAAANSUhEUgAAAGQAAABkCAYAAABw4pVUAAAF80lEQVR4nO2dTWhcVRTH33uT92YyyUy+S5ooQVs/sCbGUki1glpCUfBjk4IWwV1xJYLu3YlLFepCcCfUhUXcBIsbFSx+RCu1qFGL00idhGamyUwmeZOZ9567e+//4h0nkpl7Fue3OpcT3pvwv+ecO/dr3Hl3IXGIUnz1YWg3H6oIe/CjfvDlPvza+Jyt03PQDs6uCrtcy4Jv6D353PTid+1/2H3C6/obmZawIMRgQYjRY+WtrivtxFzCTp75FtpvHVwS9vpcDXwLlVegreb/zee3wHf1yCfCricN8E3/8rKw71g0frSOwRFCDBaEGFZSltvjCztp7Br/7unBy9D+sV4X9sWtWfC1GqJGEfa79UimuzXNly65jk04QojBghCDBSGGlRqSRJHRVzlzXNjTwVfgq8ZyiOy75mfoPHP4J2iXY2lHDtaMvmLs2IQjhBgsCDHsfFNPzGnh5lOhsGsxfovPKtnl3GenwHfYMc/27kQ+tFNO8q+24zhOlOZhL6PAghCDBSGGpRpinuH94Pj7wt6MMff7KTkzO36p/YXOyfSG0acPe+tDXEMYBRaEGHZSVgtKcZ+wRzxchCorw+D88ib4Wn2/frz/Z2iHSUrYM0EGfHuYAOgIHCHEYEGIwYIQw3oNSd13N7Tv8uUM71/NPPhKTVlf4iu/tv2OnIsbGWLHPLTlGsIALAgxWBBiWK8hxcdGoT3myfxednFHytGgKux39vCOO32cgllumAuFa3fBkCOEGiwIMaynrOqJHWiHympizsOU9frao0oLh7KtSLuYssIWE8WJ5S7KEUIMFoQYLAgxrNeQ544sQXsjln1k2MPh6eLy/cI+5OBG7L1QjtTziXXwRen//dh9gSOEGCwIMaynrEf6f4N2Q+kjobahrueP3raf62UyRl/GNQ+Ze7bbfkVH4AghBgtCDBaEGNZryBNZHHZ+viOvuhj2K+Crj7c/XeIGgdEXJupUCr5fWZS0AkcIMVgQYlhPWTq7yiY2fa1oYqrU9nO2T9wj7M34Ivgq8YDS0lIWXg7UdThCiMGCEIMFIQa5GtLnyZxe1s4GvjglzxFecA60fM7Kk7KvZV0cAjcScv+2gCOEGCwIMVgQYpBLpuq58UqCy3cv5ArC/q8akpmQh30aCa48ploe77ELRwgxWBBikEtZKtUYV/2ySvfxsjjHEW/jUt+xyRVh15Mm+KIW/bCR59uAGAUWhBgsCDGs15BiEy859pVpjlqs71qTVzdtPDsDnvx5vJ7p7ds+lT4Pa1GoXdmhEg23vyrZCThCiMGCEMN6yroUTkD73mBN2PptbyrV01Vo58+jfyglh8XqBcyO4ziDqRa74XZSZl8X4AghBgtCDBaEGNZryHJ4ENpH038L23dxymNFGSK/Mf0x+M45eEWHSqitEGY88w8AeLt2+yhHCDFYEGJYT1kXrj8A7bOzPwg70K7mKTTlUbSTvWXwffE99q1bkRza7jo4M6xuctiOMX0lQ+Z01g04QojBghCDBSGG9RpSuzwCbf9B2UduNIbAN5uRq4DXmzit8trYl9D+po7PVfGUTQ7FCGvG1ET7G7o7AUcIMVgQYrAgxLBeQ0av4C6PAU+eRR/rwTOG6sX5pRjPrBeauCqoTt172sa42JErhiVtVXKsV07P4N3Z3YEjhBgsCDGsp6yBpaLRl/dCaKsb5/q0CzL16zLUDRKBdm+yuslhxMPVxGu35HB51On+EJgjhBgsCDFYEGJYryHNworRl9NqSC1p97oMRF959JTbkg/5/eArr+eEjVc8dweOEGKwIMSwnrJ01I0Mvosf7/dwXNi3+zgk1dPbalNen1FuYFrSv7mrBDfMabEbcIQQgwUhBgtCDDs1xFM2NMe4s+TNtXlhvzuJZz5mglXjI6818JzJsUBOrWQ9rAt4JgXry+Cy8RVdgSOEGCwIMdx5d6H9n13et7cq06/aL0fXFuaEnX4JZ4ILV+VZEu8ADnO9P3HBqtknn5sM4kxw0pD9cGrqJvjSpwotPnjn4QghBgtCDBaEGP8AMRR0yNoJQhkAAAAASUVORK5CYII=\" id=\"image23e81330e0\" transform=\"scale(1 -1)translate(0 -72)\" x=\"182.882857\" y=\"-35.498357\" width=\"72\" height=\"72\"/>\n",
" </g>\n",
" <g id=\"patch_13\">\n",
" <path d=\"M 182.882857 107.498357 \n",
"L 182.882857 35.7555 \n",
"\" style=\"fill: none; stroke: #000000; stroke-width: 0.8; stroke-linejoin: miter; stroke-linecap: square\"/>\n",
" </g>\n",
" <g id=\"patch_14\">\n",
" <path d=\"M 254.625714 107.498357 \n",
"L 254.625714 35.7555 \n",
"\" style=\"fill: none; stroke: #000000; stroke-width: 0.8; stroke-linejoin: miter; stroke-linecap: square\"/>\n",
" </g>\n",
" <g id=\"patch_15\">\n",
" <path d=\"M 182.882857 107.498357 \n",
"L 254.625714 107.498357 \n",
"\" style=\"fill: none; stroke: #000000; stroke-width: 0.8; stroke-linejoin: miter; stroke-linecap: square\"/>\n",
" </g>\n",
" <g id=\"patch_16\">\n",
" <path d=\"M 182.882857 35.7555 \n",
"L 254.625714 35.7555 \n",
"\" style=\"fill: none; stroke: #000000; stroke-width: 0.8; stroke-linejoin: miter; stroke-linecap: square\"/>\n",
" </g>\n",
" <g id=\"text_3\">\n",
" <!-- trouser -->\n",
" <g transform=\"translate(197.312723 16.318125)scale(0.12 -0.12)\">\n",
" <defs>\n",
" <path id=\"DejaVuSans-73\" d=\"M 2834 3397 \n",
"L 2834 2853 \n",
"Q 2591 2978 2328 3040 \n",
"Q 2066 3103 1784 3103 \n",
"Q 1356 3103 1142 2972 \n",
"Q 928 2841 928 2578 \n",
"Q 928 2378 1081 2264 \n",
"Q 1234 2150 1697 2047 \n",
"L 1894 2003 \n",
"Q 2506 1872 2764 1633 \n",
"Q 3022 1394 3022 966 \n",
"Q 3022 478 2636 193 \n",
"Q 2250 -91 1575 -91 \n",
"Q 1294 -91 989 -36 \n",
"Q 684 19 347 128 \n",
"L 347 722 \n",
"Q 666 556 975 473 \n",
"Q 1284 391 1588 391 \n",
"Q 1994 391 2212 530 \n",
"Q 2431 669 2431 922 \n",
"Q 2431 1156 2273 1281 \n",
"Q 2116 1406 1581 1522 \n",
"L 1381 1569 \n",
"Q 847 1681 609 1914 \n",
"Q 372 2147 372 2553 \n",
"Q 372 3047 722 3315 \n",
"Q 1072 3584 1716 3584 \n",
"Q 2034 3584 2315 3537 \n",
"Q 2597 3491 2834 3397 \n",
"z\n",
"\" transform=\"scale(0.015625)\"/>\n",
" </defs>\n",
" <use xlink:href=\"#DejaVuSans-74\"/>\n",
" <use xlink:href=\"#DejaVuSans-72\" x=\"39.208984\"/>\n",
" <use xlink:href=\"#DejaVuSans-6f\" x=\"78.072266\"/>\n",
" <use xlink:href=\"#DejaVuSans-75\" x=\"139.253906\"/>\n",
" <use xlink:href=\"#DejaVuSans-73\" x=\"202.632812\"/>\n",
" <use xlink:href=\"#DejaVuSans-65\" x=\"254.732422\"/>\n",
" <use xlink:href=\"#DejaVuSans-72\" x=\"316.255859\"/>\n",
" </g>\n",
" <!-- trouser -->\n",
" <g transform=\"translate(197.312723 29.7555)scale(0.12 -0.12)\">\n",
" <use xlink:href=\"#DejaVuSans-74\"/>\n",
" <use xlink:href=\"#DejaVuSans-72\" x=\"39.208984\"/>\n",
" <use xlink:href=\"#DejaVuSans-6f\" x=\"78.072266\"/>\n",
" <use xlink:href=\"#DejaVuSans-75\" x=\"139.253906\"/>\n",
" <use xlink:href=\"#DejaVuSans-73\" x=\"202.632812\"/>\n",
" <use xlink:href=\"#DejaVuSans-65\" x=\"254.732422\"/>\n",
" <use xlink:href=\"#DejaVuSans-72\" x=\"316.255859\"/>\n",
" </g>\n",
" </g>\n",
" </g>\n",
" <g id=\"axes_4\">\n",
" <g id=\"patch_17\">\n",
" <path d=\"M 268.974286 107.498357 \n",
"L 340.717143 107.498357 \n",
"L 340.717143 35.7555 \n",
"L 268.974286 35.7555 \n",
"z\n",
"\" style=\"fill: #ffffff\"/>\n",
" </g>\n",
" <g clip-path=\"url(#pbaee48a73c)\">\n",
" <image xlink:href=\"data:image/png;base64,\n",
"iVBORw0KGgoAAAANSUhEUgAAAGQAAABkCAYAAABw4pVUAAAGHElEQVR4nO2dz28bRRTHd9e7trN24kLihiZxWiJcoNBWpEUClaq0qkDihFDDgQvqhQMS/wB/Qg9cKw5IwA2BkOBGJQSHgARBKKUEpQmlBUQbJWkcx/GPtb273GbmOyhLEmHPk3if0xs/Z3adN++92fGbsX3RvhRbhHAnxoVcey8NusqXh4Q8duXbXfe5/P6pHXXlyz/iC7HZf4dj9OrMP2CDEIMNQgzX9A3o3HpjUsiLx6+C7nLhrJDvXtl9n++c+Qja57KrQn71zJugc2bnd99xD2APIQYbhBjkQlaYkfJ6WAdd2unuq8+VTgHaf3rrQq4dzoKuMLuvS/xnsIcQgw1CDDYIMcjlkAvn54W8EaHOs7UXEkgNPyjkcuYG9mPJfmqTOCYx2/Qf9hBisEGIQS5kPTt0S8ibEa72DqTau+/o4LAQs3YHVI1YfuxWcfdhsB+whxCDDUIMNggxyOWQb6plIU8VV0HXiVNKK3kZJSwMCNmzwx37iQY4hzAJsEGIQS5kFdM1Ia91h0A34m0rLVyl1YkySliKcdy1LSX0uaRqPNhDqMEGIQYbhBjkckjBbQq5FmGe8B116SQ5h4SeHGtZG6fI98OckA8crFmUYA8hBhuEGGwQYpjPIbYNzWp3YIc37o1UIJdE6rEHuputMSF3wpRFCfYQYrBBiGE+ZGn7MYJI3tLnfx0H3SfHPhTy17kXQRfVsaguUqa9i8EY6N5deE7IzkJ+jzfcW9hDiMEGIQYbhBjmc4jGp3OnhTzyPU5JX5l5XcjOSyOgy3/8HbTXT8iq7RuNCdCVrsqPHTvB/m+2B7CHEIMNQgxyIcvbkGEqeACf4isLRSGnH8WxpE9et07KUPTD2iToOiUZznL39lB81wfYQ4jBBiEGG4QY5HKIuo0ws4HLKs2izCmdfHK1yNPlO0K+vTkMuoEtWTiXrmIOMV2Dwh5CDDYIMYyHLNvFW+j6MmjYEU57HaVEtz2Kez50XhuVT+5vr7wMulxbXsNpYMjCKuD+wx5CDDYIMdggxDCeQ5wCFlSHvixO6OS124tk7E9VsHBBp+RuCLndxn7CrByHwSguurgLyffba9hDiMEGIQYbhBjGc0h05BC0nbYyRrR1DLchn0vif9loU4/lHnffx28FgyFZjJepmF4sQdhDiMEGIYbxkNUa9aEdK6sldoThxOlIZZhJDjVTrtyPGMe4BBMqW0u2J/D4joKjFFZE/V9IYQ8hBhuEGGwQYhjPIXEK43vsKUdd2NreDeWtUS45vhdTsrJEXzqxc7KjlFYnZ3tKEV3AOeR/DxuEGMZDVsfXxoSjrvZiOOsqM+RUHrc6h89Paz3PyX5a+DFdZdqrHtxsWZblZOQLYdD/ul/2EGKwQYjBBiGG+RyS0ypLGnKMaKcqWf49uVxSmcK/W53GozYcZazFTfyYjlKwYusz24yWVPoMewgx2CDEMB6yvAau2kZKkYO/gmHJDXZe4dV/WqQStWRD+zN1qhtp/4FY217db9hDiMEGIQYbhBjGc0jlMRwTXkXmjfo45hD168RnHrkNqiefugvtvC0L6Q6MbYFu0xkUsh1iwZ09qBTONRoJd94b2EOIwQYhBhuEGMZzSOzgQ0J3XC55H528BzrHlu99YRiroosu5onZljx59IniCujKR34S8gfBWdBZzZZlEvYQYrBBiGE+ZGkz2/NHl4Q8MzIHuutNeURGqC0F/9wsQTujLOmWc/g7JCf9P4RcOFzFG0gn7zvpNewhxGCDEIMNQgzjOaRTwn3irrIZ3dMO0VfzQsm7j/3EWFRXDWWJyrn8IugWgnEhR1oSa00/LK9/Da/RD9hDiMEGIYbxkGVv4DTz1ODvQv6lhYdXVrry6XvZfiix36X6qJBP+7+B7k5LHqCpPv1blmWtnZD7RcauJV6iJ7CHEIMNQgw2CDGM55DsOo6JmfyvQr7exmMvvtp+XMgX/Jug+6J+DNrLVXmC6fIQ5hv1d642VwZRZ/iQUvYQYrBBiGFftC8ZPcrALeHUduktuWrrbeFTdCrhu6NMVfsYSlN7iLfCtOzXX8Vfiy58Ni/kqNX/L6vYQ4jBBiEGG4QYfwPOamOsbS30dQAAAABJRU5ErkJggg==\" id=\"imagef82e34f303\" transform=\"scale(1 -1)translate(0 -72)\" x=\"268.974286\" y=\"-35.498357\" width=\"72\" height=\"72\"/>\n",
" </g>\n",
" <g id=\"patch_18\">\n",
" <path d=\"M 268.974286 107.498357 \n",
"L 268.974286 35.7555 \n",
"\" style=\"fill: none; stroke: #000000; stroke-width: 0.8; stroke-linejoin: miter; stroke-linecap: square\"/>\n",
" </g>\n",
" <g id=\"patch_19\">\n",
" <path d=\"M 340.717143 107.498357 \n",
"L 340.717143 35.7555 \n",
"\" style=\"fill: none; stroke: #000000; stroke-width: 0.8; stroke-linejoin: miter; stroke-linecap: square\"/>\n",
" </g>\n",
" <g id=\"patch_20\">\n",
" <path d=\"M 268.974286 107.498357 \n",
"L 340.717143 107.498357 \n",
"\" style=\"fill: none; stroke: #000000; stroke-width: 0.8; stroke-linejoin: miter; stroke-linecap: square\"/>\n",
" </g>\n",
" <g id=\"patch_21\">\n",
" <path d=\"M 268.974286 35.7555 \n",
"L 340.717143 35.7555 \n",
"\" style=\"fill: none; stroke: #000000; stroke-width: 0.8; stroke-linejoin: miter; stroke-linecap: square\"/>\n",
" </g>\n",
" <g id=\"text_4\">\n",
" <!-- trouser -->\n",
" <g transform=\"translate(283.404152 16.318125)scale(0.12 -0.12)\">\n",
" <use xlink:href=\"#DejaVuSans-74\"/>\n",
" <use xlink:href=\"#DejaVuSans-72\" x=\"39.208984\"/>\n",
" <use xlink:href=\"#DejaVuSans-6f\" x=\"78.072266\"/>\n",
" <use xlink:href=\"#DejaVuSans-75\" x=\"139.253906\"/>\n",
" <use xlink:href=\"#DejaVuSans-73\" x=\"202.632812\"/>\n",
" <use xlink:href=\"#DejaVuSans-65\" x=\"254.732422\"/>\n",
" <use xlink:href=\"#DejaVuSans-72\" x=\"316.255859\"/>\n",
" </g>\n",
" <!-- trouser -->\n",
" <g transform=\"translate(283.404152 29.7555)scale(0.12 -0.12)\">\n",
" <use xlink:href=\"#DejaVuSans-74\"/>\n",
" <use xlink:href=\"#DejaVuSans-72\" x=\"39.208984\"/>\n",
" <use xlink:href=\"#DejaVuSans-6f\" x=\"78.072266\"/>\n",
" <use xlink:href=\"#DejaVuSans-75\" x=\"139.253906\"/>\n",
" <use xlink:href=\"#DejaVuSans-73\" x=\"202.632812\"/>\n",
" <use xlink:href=\"#DejaVuSans-65\" x=\"254.732422\"/>\n",
" <use xlink:href=\"#DejaVuSans-72\" x=\"316.255859\"/>\n",
" </g>\n",
" </g>\n",
" </g>\n",
" <g id=\"axes_5\">\n",
" <g id=\"patch_22\">\n",
" <path d=\"M 355.065714 107.498357 \n",
"L 426.808571 107.498357 \n",
"L 426.808571 35.7555 \n",
"L 355.065714 35.7555 \n",
"z\n",
"\" style=\"fill: #ffffff\"/>\n",
" </g>\n",
" <g clip-path=\"url(#p4519855026)\">\n",
" <image xlink:href=\"data:image/png;base64,\n",
"iVBORw0KGgoAAAANSUhEUgAAAGQAAABkCAYAAABw4pVUAAAJqklEQVR4nO1dSY9cVxV+Q9WrqacqD90eYnewnbblGAXkAGoJLCSDxCJZOSs2SLBhGyliC3+BvXdIrFggFiEMUoiUgShyZDLY7gG33fNgV1dX1/gmFkjvnO903rOrVBEX6Xyre/u8eve+Pu9+59xzz73Pvmnfii2D4F65lJTrrxwDWXvaScr+OP4udqxUOL6oB1QefxyBbOqdB0k5rNef0dvRI+MxFP8LqEIMgyrEMORGdifHpXIUomgcCX/x11eT8jdfXQbZhbGlpPzuho1tBNRGtdgD0faTSahXJ1tJudkugOzEVJPar62DbOWXZLfW6lfwnrfxOYp/+tgaNXSEGAZViGGwR+X22jlivzgIQLZw+zrUHY8ozVsuoYwxkYPMZxWeUlf3L2O37QjpLThOvm550UNZmX7rHeDvOjPkBtcuPwFZtdjBvr72NClH7TZ2NoPCs6AjxDCoQgyDKsQwjMztlXaDo7iKHB56xOG2L68mHF5EYbiaT8rlDcn9whT6JJdhlWCMro3zKAsrZEM6fRS2uvgcZ9trX91xy7KsOEqXZUBHiGFQhRiG4SnLRsqwXXLzJH1NLeDw3Z4nyijsuyCLWLWyLCiDUVjpEcrywn0NS/Su2YLN3C5d6/RFNCBPfS3m8Tn2VqesVIj/hxUPN5vQEWIYVCGGQRViGIa3IYIjs9ze6jsLUN/6wcXUa8MS3TffRF4uPyS70T4r2hMUbvfZu5ZhQ4IKCt0S3ffbJ1dBdvf3tdR+D2szJHSEGAZViGEY2UzdrVaTsn/1PAo/+Ayq+TpzkQXVOGyG3T2BNFDeJJn3VLjLGTNuSWc86SEsYBv8Dd3tjoGs9s9tqNtnz1D79X3sT6tlDQMdIYZBFWIYVCGGYSAbsvnmfFJuXu2DbPrUflL+3vTnILv/i8tQn/g3ldsnRYcY9fpI4Vaf5TG4PTQMjvCC4xzJbbFg5/KuO2hDZmoHSfnunQsgmz2H0ee//+52Uv7Op2+AzGcJGd076C6f+80HVhp0hBgGVYhhUIUYhoFsSFBmlR7qcnuFeHJtfB9ke9cnoJ5vEW/7mN9mFTHRA8Ho3ukLkZtel+GRyGOriS7Kcg7NXyqP8Rl3v4UJdy9/9NOk3D5E2flT9CArM+lhJQkdIYZBFWIYBqKsk3fI7VurIkfkD0i3C6dPgCyuoos6/WeKom7Pn8FreYKZWOrj7qukKOna8roditVEHi7xcDWz4BK9nPy0C7L1G0WoT7L84tZuGWT7bUoAPHZHdDYDOkIMgyrEMKhCDMNANqS4Q8nGUQ7jGi6j28MDTKDOVdEWBKuUYJY7fAEbYXSfFfLoTwh3tY12gifHuWgKwP7YebQhLnN7c4sYbu/9bBrqzQ65unYX7cTrs7Tk8NeD71vPCx0hhkEVYhgGWzFkyWBxBWefkUd5r/lVnLW6c02ss9XFKMMjlJQF25vFKmCITVpRiajH7eJ7FxY5L+Lv1hsUOjgzgR24MbcI9X88oC3csaC+POu819T9If+3UIUYBlWIYRjIhrh7tJpmBegChkW2b6+BxHz11AbUm1Xi6Sx31QkEwTOalqGT3CHWnY6Tem2UY/tDfHwnWy0Kj9gNdHuvT6xA/T2HJfyJ8IzPGi1tYAZK1s4RHSGGQRViGAairHBtk9WQsuKMGfZ4Dk9daPrkMh9NTqCyXIQKWbA1KOFMvfAEKYPn78okOqAwkZLrsJm6P4sZGMWM/XdxAYnIYTe2H23Ky1OhI8QwqEIMgyrEMAxkQ2KfSN32xR5DbjcEL1eEDenPnki9FjqHJ1lYAQsiZ7nL/+1QStlCOyWzvaOIblSfwxXCTX8K78OP8xBu70aPXPtBTqbTEWIYVCGGQRViGIbesOOIkDb37WNx10jw9N41MgaumGtEfB4i5yisSRmeOTLXYGIZ4o/yzHBl2LDGJaxv9URWH29EZK88OuQJ1nhqXRZ0hBgGVYhhGJqy8i3pdtLYDzBnzPpw60WoN75LWQeVz9G15JA0xOnFRU/6SLimzxIrHOGS8r0l0Thylleg8EjlGkZp9/oVbIQ9s+MiZTV69FyY2ZwNHSGGQRViGFQhhmFoG1LeRO7l+W5BBflUnsxmNakuQx7cDQ7SzcvR3x2xKWQn/DHsKw/Nh0Vx6DOzC9NjmC0j3fexCRHbYdj7jEL3E9Zy6nUSOkIMgyrEMAxNWeOPcRrduEi3iryM6a9lWeU1mjofmakzdpMzfg65ZfpIHnCXl+X+ECrnPXwOntu7tHMcZDdfxFONHubonPiKhw/SXJr6qm4/EzpCDIMqxDCoQgzD0Dakcg+TyMLXZ5Ky3NPXaeMBxDzXmR+wb1mW1auxIzFEtDdit4me0XPuoZZ20110v4834m5vv4vu+vnSHtTf7VOiXK2Eh/G7y8I4Pid0hBgGVYhhGJqygpXH4i9EWW4L9Ww3ccodMbdTRnRh67PoHZ+dy5PgyttIkz2+FVscUMlPrbO3cGNJ8SWink4dt+YttGasNAQidOC9/0VSHuQUeB0hhkEVYhhUIYZhZKeSltbpVr7Ysnzk9De+vXkyI+FNbg9h9qb2JbaxdQNjJzbbWxK78sB/vpcF38n6Ln0azy2j3/3h+izUeWL23OQOyJa7Yi/2c0JHiGFQhRgGVYhhGJkNOfYlcfj6D1Hm1ZHD+fcJA3T1YeVPJrjl2QKeXDH8+fx7UP/bFp2Eulk/BbLwG8Tv7j3sQH6HDFVwGmcQPRFKKZWps2/ffRlkL1mfWMNAR4hhUIUYhsEoK+NzouNf0KGP8U08OFh+QtVn+WaO2LbHQyfBpAyPUHnnxxhN/aSOHwBodChcI7dX+6zencbOuW32jjaRosIxdIPLU9R5518Y0QYM8H0qHSGGQRViGFQhhmEwG5LxOdHwwVJSLq/Ng6wjPszC93bIPSB8L3ppB7m3S0ke1uVzWyC7v417yvlqX3waDVXtfQq5t3+EZ3LE9yidJXbwfXWO430aLXKZz/02/YD9Qb5PpSPEMKhCDIN907413PfeBnDl1n+FFNa6wIa+OHu9skjuoy/ObC9c20/KXTFrHitjcu8hO9XHb6JLanfIfXeO4+/CQ2Lx0+fxIPqDDq58vvAW5faGSw+tUUBHiGFQhRgGVYhhGMyGSLuRhme4ee4xCq3svTYHsiev0G9v3fgIZG8/usKawL5Uy7hX49LkblJ+sI8u8VSRrvWE333/L7QXevYPuyAL7+GppF8HdIQYBlWIYRiMsjKivZl0NuSXlHs/eRXqQYXen9ATJzm46e3LQ5bLu9T30h8/HqpvXxd0hBgGVYhhUIUYhv8AY1XqYCPMS2EAAAAASUVORK5CYII=\" id=\"image665b50bd2e\" transform=\"scale(1 -1)translate(0 -72)\" x=\"355.065714\" y=\"-35.498357\" width=\"72\" height=\"72\"/>\n",
" </g>\n",
" <g id=\"patch_23\">\n",
" <path d=\"M 355.065714 107.498357 \n",
"L 355.065714 35.7555 \n",
"\" style=\"fill: none; stroke: #000000; stroke-width: 0.8; stroke-linejoin: miter; stroke-linecap: square\"/>\n",
" </g>\n",
" <g id=\"patch_24\">\n",
" <path d=\"M 426.808571 107.498357 \n",
"L 426.808571 35.7555 \n",
"\" style=\"fill: none; stroke: #000000; stroke-width: 0.8; stroke-linejoin: miter; stroke-linecap: square\"/>\n",
" </g>\n",
" <g id=\"patch_25\">\n",
" <path d=\"M 355.065714 107.498357 \n",
"L 426.808571 107.498357 \n",
"\" style=\"fill: none; stroke: #000000; stroke-width: 0.8; stroke-linejoin: miter; stroke-linecap: square\"/>\n",
" </g>\n",
" <g id=\"patch_26\">\n",
" <path d=\"M 355.065714 35.7555 \n",
"L 426.808571 35.7555 \n",
"\" style=\"fill: none; stroke: #000000; stroke-width: 0.8; stroke-linejoin: miter; stroke-linecap: square\"/>\n",
" </g>\n",
" <g id=\"text_5\">\n",
" <!-- shirt -->\n",
" <g transform=\"translate(377.523393 16.318125)scale(0.12 -0.12)\">\n",
" <defs>\n",
" <path id=\"DejaVuSans-68\" d=\"M 3513 2113 \n",
"L 3513 0 \n",
"L 2938 0 \n",
"L 2938 2094 \n",
"Q 2938 2591 2744 2837 \n",
"Q 2550 3084 2163 3084 \n",
"Q 1697 3084 1428 2787 \n",
"Q 1159 2491 1159 1978 \n",
"L 1159 0 \n",
"L 581 0 \n",
"L 581 4863 \n",
"L 1159 4863 \n",
"L 1159 2956 \n",
"Q 1366 3272 1645 3428 \n",
"Q 1925 3584 2291 3584 \n",
"Q 2894 3584 3203 3211 \n",
"Q 3513 2838 3513 2113 \n",
"z\n",
"\" transform=\"scale(0.015625)\"/>\n",
" <path id=\"DejaVuSans-69\" d=\"M 603 3500 \n",
"L 1178 3500 \n",
"L 1178 0 \n",
"L 603 0 \n",
"L 603 3500 \n",
"z\n",
"M 603 4863 \n",
"L 1178 4863 \n",
"L 1178 4134 \n",
"L 603 4134 \n",
"L 603 4863 \n",
"z\n",
"\" transform=\"scale(0.015625)\"/>\n",
" </defs>\n",
" <use xlink:href=\"#DejaVuSans-73\"/>\n",
" <use xlink:href=\"#DejaVuSans-68\" x=\"52.099609\"/>\n",
" <use xlink:href=\"#DejaVuSans-69\" x=\"115.478516\"/>\n",
" <use xlink:href=\"#DejaVuSans-72\" x=\"143.261719\"/>\n",
" <use xlink:href=\"#DejaVuSans-74\" x=\"184.375\"/>\n",
" </g>\n",
" <!-- shirt -->\n",
" <g transform=\"translate(377.523393 29.7555)scale(0.12 -0.12)\">\n",
" <use xlink:href=\"#DejaVuSans-73\"/>\n",
" <use xlink:href=\"#DejaVuSans-68\" x=\"52.099609\"/>\n",
" <use xlink:href=\"#DejaVuSans-69\" x=\"115.478516\"/>\n",
" <use xlink:href=\"#DejaVuSans-72\" x=\"143.261719\"/>\n",
" <use xlink:href=\"#DejaVuSans-74\" x=\"184.375\"/>\n",
" </g>\n",
" </g>\n",
" </g>\n",
" <g id=\"axes_6\">\n",
" <g id=\"patch_27\">\n",
" <path d=\"M 441.157143 107.498357 \n",
"L 512.9 107.498357 \n",
"L 512.9 35.7555 \n",
"L 441.157143 35.7555 \n",
"z\n",
"\" style=\"fill: #ffffff\"/>\n",
" </g>\n",
" <g clip-path=\"url(#p5126b7e563)\">\n",
" <image xlink:href=\"data:image/png;base64,\n",
"iVBORw0KGgoAAAANSUhEUgAAAGQAAABkCAYAAABw4pVUAAAHCElEQVR4nO2dXWwUVRTHZ2ZnP7vb3S4ttFCEEktBgggEE0KABElE/IghmugDCZiIL8ZEX40P+mDii9FEjTEmvhgJxi+UYAyIiUAUI6iIQKsS3ULod7v92I/Ozqxv98x/zKyltjsn6fk9nduzmTvNmXvPnXPPPaPv1h+pakzRN6+DdveTDaSzdNCtPFqBdvjked/rjjyxVclD26dBZ+TDSl72jQO6+NEf/uOO/z/GvPcg3BJiEGaIQZhhBn0Dtbj+PLaPbXpdyVNVvPX9+WegvfJkjQvvG1biuQ3vgarPDin50bZDeM2jte52bpARwgwxCDOCmbJ015K16r/q3tTWC+2P8puV3BEdAN1tX5Vm3H1+MqZk9xSlaZr2+/RiJVuD8Rlfc66QEcIMMQgzxCDMCMSH6CGat6sVDHmU79+i5F1NuM7st9JKPjveCTrj9E8z7j+TKiq5r5ICXV+F+tAzGFapBzJCmCEGYUYgU1bVtn11ufvoGbGr+Lw0mxNK/rB/I+q0nhn3vzQ5ruQROwm6u2J/032ORmZ8zblCRggzxCDMEIMwI5jQSY1wyWPbvlNyqRoGXauZp0scXzTr7pcm6DrTVQydWO4osoO7kvVARggzxCDMCHyDKtS5Ctpr4t8quafUBrqYbil5ydlR0GE6Qm1SJkWGC04UdGNOgu6tLFPWgkcMwgwxCDMC9yFXnm2GdiZUUHJYxxDL1SL5FOfi1Vn32REd9NWVHFpqVzIV39/NFzJCmCEGYYYYhBmB+5Ddm3+D9qXiciVnzSnQtUdGlHxRa511n/tSFKp/qW8X6BpN2k2U0IkgBuFG4FPWzWIjtBvMspJXeZLhXr6yR8lt2pVZ9zlo03N4/DKeQelspz6NUv2fVxkhzBCDMEMMwoxgEuVM6jZhYjJa0abQRcIogy55OK35oYcxQ6Rq0XXN9mWgO124XcnhXgy/D2Up/O7EbiWoPzfICGGGGIQZgUxZ/U/dreSsjsvXqEER1j/K+Dbe+MkFJf8rTaLqP71MbVgK7QeSR5T8queRdBz6gzmOCRD1QEYIM8QgzBCDMCMQHzK2jvzEcKkBdKZBu4QHl1wC3cmO/Uq2e/7Ei4Y8873r3EluLz53h8c3UH9TGNEdG6Tk67B/Tvi8ISOEGWIQZgQT7Q3TonV8Gt+Uy3ZWyZZncTu2sUXJKc+UVbX8ExI6um5CO1+ht3Er5VlAh6htZeo/Z8kIYYYYhBliEGYE4kPCSYrEJsIW6CxXqYv38+tBN/AQRX9TRzTE8Z/v97RiIsWQRUehK4vQ93St6FNy7tQK32vOFzJCmCEGYYYYhBmB+JCWJjpv3hQtgG6gQPN7rpwF3StbPlbyOxoe9KnFzgZMzP48v0nJyWZMxluT7ldyryM+ZMEjBmFGIFNWzKSlZmtsAnRDRYq2DpSwUs/OxRQCeffOvaCrdV5kYwSfu8+qFOFdls57f64IzbxI3ZwhI4QZYhBmiEGYEYgPGZykXcIdLehDdJ0yTUo23t6EQ6Hx7udwp7HzAPZh7aYKpmH9Z9S5ymlEQhhySbvOh1Trn3QiI4QbYhBmBDJlTV2jHN2utbib96V1h5LbEhiJPVFYreTXth0G3ZvaamjnDlLinO1Jois7/v92wqBItExZghiEG2IQZgTiQ9I9FLpY/vAw6MYLVCg/nsXdxO4CLYkfbPFWIUUfcmA9VaYbsDGi7C7ZMeHJejF08jcOFrSrCzJCmCEGYUYgU1b2KiUrNOg4LTUm/EOshk5v6qeKuHmUPoNFMXckv1Zyt4VHrxtdFeXiJvYf02mpbUfr/0VBGSHMEIMwQwzCjEB8SDRHVX3aTf8kaUfDsxuLI1RE3/0tEU3TtBfbv4D2oKu6aM7CZIlRV7K17eAz6T6KbeOKuC7ICGGGGIQZYhBmBOJDKtf+UnLJU5h/bZYS1QzPgR13yCNl4PvKsUlMzE6HipofUxVyDmXPrmSLSX7KiUppjQWPGIQZgVeUe3tkK7Qfb/5eyW/dwAKV+TgtV2OecyXLwyPQdn/+ruwJ27qr1rUnx0AXcU2L8ZtSWmPBIwZhhhiEGYH7kE+PbIf2C09TCaazmeugixnkN7zV5k6MYnVRd4mOtQ2Y2dIcnlSyewmsaZp2ZpJ2HlvPYR/1QEYIM8QgzAh8ylr5QS+0fzlE8m0RTIDI266SGFW89UwYExnclem8b/UD07SDmJtsAt2vw/SNkvSp87VufV6QEcIMMQgzxCDMCNyHVKdw7g+5Irzebwy6o73ub1VpmqbtbMQzhoMV8hPeT7i6s1fuXXIZdD/mKZsFPVh9kBHCDDEIMwKfsuwhnBje6L9Hye468JqmaePTlPebieAG1IUb7dCOR+mtvqsZv0PS76oWETHwSFvIcG9K4edd64GMEGaIQZghBmHGP08yyeNAr6LhAAAAAElFTkSuQmCC\" id=\"image840c39e69c\" transform=\"scale(1 -1)translate(0 -72)\" x=\"441.157143\" y=\"-35.498357\" width=\"72\" height=\"72\"/>\n",
" </g>\n",
" <g id=\"patch_28\">\n",
" <path d=\"M 441.157143 107.498357 \n",
"L 441.157143 35.7555 \n",
"\" style=\"fill: none; stroke: #000000; stroke-width: 0.8; stroke-linejoin: miter; stroke-linecap: square\"/>\n",
" </g>\n",
" <g id=\"patch_29\">\n",
" <path d=\"M 512.9 107.498357 \n",
"L 512.9 35.7555 \n",
"\" style=\"fill: none; stroke: #000000; stroke-width: 0.8; stroke-linejoin: miter; stroke-linecap: square\"/>\n",
" </g>\n",
" <g id=\"patch_30\">\n",
" <path d=\"M 441.157143 107.498357 \n",
"L 512.9 107.498357 \n",
"\" style=\"fill: none; stroke: #000000; stroke-width: 0.8; stroke-linejoin: miter; stroke-linecap: square\"/>\n",
" </g>\n",
" <g id=\"patch_31\">\n",
" <path d=\"M 441.157143 35.7555 \n",
"L 512.9 35.7555 \n",
"\" style=\"fill: none; stroke: #000000; stroke-width: 0.8; stroke-linejoin: miter; stroke-linecap: square\"/>\n",
" </g>\n",
" <g id=\"text_6\">\n",
" <!-- trouser -->\n",
" <g transform=\"translate(455.587009 16.318125)scale(0.12 -0.12)\">\n",
" <use xlink:href=\"#DejaVuSans-74\"/>\n",
" <use xlink:href=\"#DejaVuSans-72\" x=\"39.208984\"/>\n",
" <use xlink:href=\"#DejaVuSans-6f\" x=\"78.072266\"/>\n",
" <use xlink:href=\"#DejaVuSans-75\" x=\"139.253906\"/>\n",
" <use xlink:href=\"#DejaVuSans-73\" x=\"202.632812\"/>\n",
" <use xlink:href=\"#DejaVuSans-65\" x=\"254.732422\"/>\n",
" <use xlink:href=\"#DejaVuSans-72\" x=\"316.255859\"/>\n",
" </g>\n",
" <!-- trouser -->\n",
" <g transform=\"translate(455.587009 29.7555)scale(0.12 -0.12)\">\n",
" <use xlink:href=\"#DejaVuSans-74\"/>\n",
" <use xlink:href=\"#DejaVuSans-72\" x=\"39.208984\"/>\n",
" <use xlink:href=\"#DejaVuSans-6f\" x=\"78.072266\"/>\n",
" <use xlink:href=\"#DejaVuSans-75\" x=\"139.253906\"/>\n",
" <use xlink:href=\"#DejaVuSans-73\" x=\"202.632812\"/>\n",
" <use xlink:href=\"#DejaVuSans-65\" x=\"254.732422\"/>\n",
" <use xlink:href=\"#DejaVuSans-72\" x=\"316.255859\"/>\n",
" </g>\n",
" </g>\n",
" </g>\n",
" </g>\n",
" <defs>\n",
" <clipPath id=\"p8a44a1479b\">\n",
" <rect x=\"10.7\" y=\"35.7555\" width=\"71.742857\" height=\"71.742857\"/>\n",
" </clipPath>\n",
" <clipPath id=\"p9f0c7898d2\">\n",
" <rect x=\"96.791429\" y=\"35.7555\" width=\"71.742857\" height=\"71.742857\"/>\n",
" </clipPath>\n",
" <clipPath id=\"pfd11250cbb\">\n",
" <rect x=\"182.882857\" y=\"35.7555\" width=\"71.742857\" height=\"71.742857\"/>\n",
" </clipPath>\n",
" <clipPath id=\"pbaee48a73c\">\n",
" <rect x=\"268.974286\" y=\"35.7555\" width=\"71.742857\" height=\"71.742857\"/>\n",
" </clipPath>\n",
" <clipPath id=\"p4519855026\">\n",
" <rect x=\"355.065714\" y=\"35.7555\" width=\"71.742857\" height=\"71.742857\"/>\n",
" </clipPath>\n",
" <clipPath id=\"p5126b7e563\">\n",
" <rect x=\"441.157143\" y=\"35.7555\" width=\"71.742857\" height=\"71.742857\"/>\n",
" </clipPath>\n",
" </defs>\n",
"</svg>\n"
],
"text/plain": [
"<Figure size 900x150 with 6 Axes>"
]
},
"metadata": {},
"output_type": "display_data"
}
],
"source": [
"d2l.predict_ch3(net, test_iter)"
]
},
{
"cell_type": "markdown",
"id": "c97420c6",
"metadata": {
"origin_pos": 32
},
"source": [
"## 小结\n",
"\n",
"* 手动实现一个简单的多层感知机是很容易的。然而如果有大量的层，从零开始实现多层感知机会变得很麻烦（例如，要命名和记录模型的参数）。\n"
]
}
],
"metadata": {
"kernelspec": {
"display_name": "Python 3 (ipykernel)",
"language": "python",
"name": "python3"
},
"language_info": {
"codemirror_mode": {
"name": "ipython",
"version": 3
},
"file_extension": ".py",
"mimetype": "text/x-python",
"name": "python",
"nbconvert_exporter": "python",
"pygments_lexer": "ipython3",
"version": "3.9.17"
},
"required_libs": []
},
"nbformat": 4,
"nbformat_minor": 5
}
前馈神经网络shr/实例代码/softmax-regression-scratch.ipynb 0 → 100644
This source diff could not be displayed because it is too large. You can view the blob instead.
卷积神经网络wyx/CNN_codes/.idea/.gitignore 0 → 100644
# Default ignored files
# Default ignored files
/shelf/
/workspace.xml
# Editor-based HTTP Client requests
/httpRequests/
# Datasource local storage ignored files
/dataSources/
/dataSources.local.xml
卷积神经网络wyx/CNN_codes/.idea/.name 0 → 100644
cnn_net.py
cnn_net.py
\ No newline at end of file
卷积神经网络wyx/CNN_codes/.idea/CNN.iml 0 → 100644
<?xml version="1.0" encoding="UTF-8"?>
<?xml version="1.0" encoding="UTF-8"?>
<module type="PYTHON_MODULE" version="4">
<component name="NewModuleRootManager">
<content url="file://$MODULE_DIR$" />
<orderEntry type="inheritedJdk" />
<orderEntry type="sourceFolder" forTests="false" />
</component>
</module>
\ No newline at end of file
卷积神经网络wyx/CNN_codes/.idea/inspectionProfiles/profiles_settings.xml 0 → 100644
<component name="InspectionProjectProfileManager">
<component name="InspectionProjectProfileManager">
<settings>
<option name="USE_PROJECT_PROFILE" value="false" />
<version value="1.0" />
</settings>
</component>
\ No newline at end of file
卷积神经网络wyx/CNN_codes/.idea/misc.xml 0 → 100644
<?xml version="1.0" encoding="UTF-8"?>
<?xml version="1.0" encoding="UTF-8"?>
<project version="4">
<component name="ProjectRootManager" version="2" project-jdk-name="pytorch" project-jdk-type="Python SDK" />
</project>
\ No newline at end of file
卷积神经网络wyx/CNN_codes/.idea/modules.xml 0 → 100644
<?xml version="1.0" encoding="UTF-8"?>
<?xml version="1.0" encoding="UTF-8"?>
<project version="4">
<component name="ProjectModuleManager">
<modules>
<module fileurl="file://$PROJECT_DIR$/.idea/CNN.iml" filepath="$PROJECT_DIR$/.idea/CNN.iml" />
</modules>
</component>
</project>
\ No newline at end of file
卷积神经网络wyx/CNN_codes/cnn_net.py 0 → 100644
import torch
import torch
from torch import nn
class Net(nn.Module):
def __init__(self):
super().__init__()
self.module = nn.Sequential(
nn.Conv2d(3,32,5,1,2),
# 对于所有的batch中的同一个channel的数据元素进行标准化处理
nn.BatchNorm2d(32),
nn.ReLU(),
nn.MaxPool2d(2),
nn.Conv2d(32,32,5,1,2),
nn.BatchNorm2d(32),
nn.ReLU(),
nn.MaxPool2d(2),
nn.Conv2d(32,64,5,1,2),
nn.BatchNorm2d(64),
nn.ReLU(),
#二维特征图展平
nn.Flatten(),
#64*8*8
nn.Linear(4096,1024),
nn.Linear(1024, 64),
nn.Linear(64,10),
nn.Softmax(dim=1)
)
def forward(self,input):
output = self.module(input)
return output
if __name__ == "__main__":
input = torch.ones((64,3,32,32))
net = Net()
output = net(input)
print(output.shape)
卷积神经网络wyx/CNN_codes/main.py 0 → 100644
# This is a sample Python script.
# This is a sample Python script.
# Press Shift+F10 to execute it or replace it with your code.
# Press Double Shift to search everywhere for classes, files, tool windows, actions, and settings.
def print_hi(name):
# Use a breakpoint in the code line below to debug your script.
print(f'Hi, {name}') # Press Ctrl+F8 to toggle the breakpoint.
# Press the green button in the gutter to run the script.
if __name__ == '__main__':
print_hi('PyCharm')
# See PyCharm help at https://www.jetbrains.com/help/pycharm/
卷积神经网络wyx/CNN_codes/test.py 0 → 100644
from PIL import Image
from PIL import Image
import torch
from cnn_net import Net
from torchvision import transforms
img_path = "./dataset/img.png"
image = Image.open(img_path)
image = image.convert("RGB")
compose = transforms.Compose([transforms.Resize((32,32)),
transforms.ToTensor()
])
image = compose(image)
net = Net()
net.load_state_dict(torch.load("finaly_net100.pt"))
input = torch.reshape(image,[1,3,32,32])
#{'airplane': 0, 'automobile': 1, 'bird': 2, 'cat': 3, 'deer': 4, 'dog': 5, 'frog': 6, 'horse': 7, 'ship': 8, 'truck': 9}
output = net(input)
print(output)
print(output.argmax(1))
卷积神经网络wyx/CNN_codes/train.py 0 → 100644
import torch
import torch
import torchvision
import cnn_net
from torch.utils.tensorboard import SummaryWriter
from torch.utils.data import DataLoader
#32*32*3 10种类别
dataset_train = torchvision.datasets.CIFAR10(root="./dataset",train=True,transform=torchvision.transforms.ToTensor(),download=True)
dataset_test = torchvision.datasets.CIFAR10(root="./dataset",train=False,transform=torchvision.transforms.ToTensor(),download=True)
dataloder_train = DataLoader(dataset_train,64)
dataloder_test = DataLoader(dataset_test,64)
train_len = dataset_train.__len__()
test_len = dataset_test.__len__()
print(f"训练集的大小为{train_len}")
print(f"测试集的大小为{test_len}")
net = cnn_net.Net()
if torch.cuda.is_available():
net = net.cuda()
#优化器
learningrate = 1e-2
#1e-2 = 1/10^-2
optim = torch.optim.SGD(net.parameters(),lr=learningrate)
#损失值
loss = torch.nn.CrossEntropyLoss()
if torch.cuda.is_available():
loss = loss.cuda()
#训练轮次
epoh = 100
#训练次数
num_train = 0
#测试次数
num_test = 0
#绘制图像
writer = SummaryWriter("finally")
best_loss = 0
for i in range(epoh):
#训练开始
print(f"------第{i+1}轮开始--------")
for data in dataloder_train:
img,targart = data
if torch.cuda.is_available():
img = img.cuda()
targart = targart.cuda()
output = net(img)
#优化模型
optim.zero_grad()
train_loss = loss(output,targart)
train_loss.backward()
optim.step()
#优化表示
num_train += 1
if num_train%100 == 0:
print(f"第{num_train}次训练，loss的值为{train_loss}")
writer.add_scalar("train",train_loss,num_train)
#进行测试
total_loss = 0
total_accuracy = 0
with torch.no_grad():
for data in dataloder_test:
img,targart = data
if torch.cuda.is_available():
img = img.cuda()
targart = targart.cuda()
output = net(img)
total_loss += loss(output,targart)
#计算每轮分类正确的个数
total_accuracy += (output.argmax(1) == targart).sum()
num_test += 1
writer.add_scalar("test", total_loss, num_test)
print(f"第{num_test}轮时整体的loss值为{total_loss}")
print(f"第{num_test}轮时整体正确率为{total_accuracy/test_len}")
#每轮结束对模型进行保存
if i == 0 :
best_loss = total_loss
elif best_loss > total_loss:
best_loss = total_loss
torch.save(net.state_dict(), f"finaly_net100.pt")
writer.close()
\ No newline at end of file
常见的CNN模型/DenseNet_lt/Densely_Connected_Convolutional_Networks.pdf 0 → 100644
This source diff could not be displayed because it is too large. You can view the blob instead.
常见的CNN模型/DenseNet_lt/code/DenseNet_Implement.ipynb 0 → 100644
{
{
"cells": [
{
"cell_type": "code",
"execution_count": 2,
"id": "7dba137d-5ccf-422c-8e05-2ecea198b1ff",
"metadata": {},
"outputs": [],
"source": [
"import torch\n",
"from torch import nn"
]
},
{
"cell_type": "code",
"execution_count": 4,
"id": "97be88d1-1dfa-4f52-9600-ce62a66c821e",
"metadata": {},
"outputs": [],
"source": [
"def conv_block(in_channel, out_channel):\n",
" layer = nn.Sequential(\n",
" nn.BatchNorm2d(in_channel),\n",
" nn.ReLU(),\n",
" nn.Conv2d(in_channel, out_channel, kernel_size=3, padding=1, bias=False)\n",
" )\n",
" return layer\n",
"\n",
"class dense_block(nn.Module):\n",
" def __init__(self, in_channel, growth_rate, num_layers):\n",
" super(dense_block, self).__init__()\n",
" block = []\n",
" channel = in_channel\n",
" for i in range(num_layers):\n",
" block.append(conv_block(channel, growth_rate))\n",
" channel += growth_rate\n",
" self.net = nn.Sequential(*block)\n",
" def forward(self, x):\n",
" for layer in self.net:\n",
" out = layer(x)\n",
" x = torch.cat((out, x), dim=1)\n",
" return x\n",
"def transition(in_channel, out_channel):\n",
" trans_layer = nn.Sequential(\n",
" nn.BatchNorm2d(in_channel),\n",
" nn.ReLU(),\n",
" nn.Conv2d(in_channel, out_channel, 1),\n",
" nn.AvgPool2d(2, 2)\n",
" )\n",
" return trans_layer\n"
]
},
{
"cell_type": "code",
"execution_count": 5,
"id": "bf8568f2-7988-4ff0-a974-4ddb685e8886",
"metadata": {},
"outputs": [],
"source": [
"class densenet(nn.Module):\n",
" def __init__(self, in_channel, num_classes, growth_rate=32, block_layers=[6, 12, 24, 16]):\n",
" super(densenet, self).__init__()\n",
" self.block1 = nn.Sequential(\n",
" nn.Conv2d(in_channel, 64, 7, 2, 3),\n",
" nn.BatchNorm2d(64),\n",
" nn.ReLU(True),\n",
" nn.MaxPool2d(3, 2, padding=1)\n",
" )\n",
" self.DB1 = self._make_dense_block(64, growth_rate,num=block_layers[0])\n",
" self.TL1 = self._make_transition_layer(256)\n",
" self.DB2 = self._make_dense_block(128, growth_rate, num=block_layers[1])\n",
" self.TL2 = self._make_transition_layer(512)\n",
" self.DB3 = self._make_dense_block(256, growth_rate, num=block_layers[2])\n",
" self.TL3 = self._make_transition_layer(1024)\n",
" self.DB4 = self._make_dense_block(512, growth_rate, num=block_layers[3])\n",
" self.global_average = nn.Sequential(\n",
" nn.BatchNorm2d(1024),\n",
" nn.ReLU(),\n",
" nn.AdaptiveAvgPool2d((1,1)),\n",
" )\n",
" self.classifier = nn.Linear(1024, num_classes)\n",
" def forward(self, x):\n",
" x = self.block1(x)\n",
" x = self.DB1(x)\n",
" x = self.TL1(x)\n",
" x = self.DB2(x)\n",
" x = self.TL2(x)\n",
" x = self.DB3(x)\n",
" x = self.TL3(x)\n",
" x = self.DB4(x)\n",
" x = self.global_average(x)\n",
" x = x.view(x.shape[0], -1)\n",
" x = self.classifier(x)\n",
" return x\n",
"\n",
" def _make_dense_block(self,channels, growth_rate, num):\n",
" block = []\n",
" block.append(dense_block(channels, growth_rate, num))\n",
" channels += num * growth_rate\n",
"\n",
" return nn.Sequential(*block)\n",
" def _make_transition_layer(self,channels):\n",
" block = []\n",
" block.append(transition(channels, channels // 2))\n",
" return nn.Sequential(*block)\n"
]
},
{
"cell_type": "code",
"execution_count": 6,
"id": "7e93c6c0-a485-4da0-a8d2-e21341b9890f",
"metadata": {},
"outputs": [
{
"name": "stdout",
"output_type": "stream",
"text": [
"block1 output shape: torch.Size([1, 64, 56, 56])\n",
"DB1 output shape: torch.Size([1, 256, 56, 56])\n",
"TL1 output shape: torch.Size([1, 128, 28, 28])\n",
"DB2 output shape: torch.Size([1, 512, 28, 28])\n",
"TL2 output shape: torch.Size([1, 256, 14, 14])\n",
"DB3 output shape: torch.Size([1, 1024, 14, 14])\n",
"TL3 output shape: torch.Size([1, 512, 7, 7])\n",
"DB4 output shape: torch.Size([1, 1024, 7, 7])\n",
"global_average output shape: torch.Size([1, 1024, 1, 1])\n",
"classifier output shape: torch.Size([1, 10])\n"
]
}
],
"source": [
"net = densenet(3,10)\n",
"x = torch.rand(1,3,224,224)\n",
"for name,layer in net.named_children():\n",
" if name != \"classifier\":\n",
" x = layer(x)\n",
" print(name, 'output shape:', x.shape)\n",
" else:\n",
" x = x.view(x.size(0), -1)\n",
" x = layer(x)\n",
" print(name, 'output shape:', x.shape)\n"
]
},
{
"cell_type": "code",
"execution_count": null,
"id": "4515798e-d5c8-4267-a11b-3291e0afd69c",
"metadata": {},
"outputs": [],
"source": []
}
],
"metadata": {
"kernelspec": {
"display_name": "Python 3 (ipykernel)",
"language": "python",
"name": "python3"
},
"language_info": {
"codemirror_mode": {
"name": "ipython",
"version": 3
},
"file_extension": ".py",
"mimetype": "text/x-python",
"name": "python",
"nbconvert_exporter": "python",
"pygments_lexer": "ipython3",
"version": "3.9.12"
}
},
"nbformat": 4,
"nbformat_minor": 5
}
常见的CNN模型/DenseNet_lt/readme.txt 0 → 100644
PPT部分文字不多，大家可以多看下演讲者备注里，有写很多细节。
PPT部分文字不多，大家可以多看下演讲者备注里，有写很多细节。
另外不懂的也可以单独来问我。
\ No newline at end of file
常见的CNN模型/GoogleNet_gcq/code/googlenet.ipynb 0 → 100644
{
{
"cells": [
{
"cell_type": "markdown",
"id": "e0e5a226",
"metadata": {
"id": "e0e5a226"
},
"source": [
"The following additional libraries are needed to run this\n",
"notebook. Note that running on Colab is experimental, please report a Github\n",
"issue if you have any problem."
]
},
{
"cell_type": "code",
"execution_count": 10,
"id": "82e98083",
"metadata": {
"execution": {
"iopub.execute_input": "2023-08-18T07:18:45.698938Z",
"iopub.status.busy": "2023-08-18T07:18:45.698048Z",
"iopub.status.idle": "2023-08-18T07:18:49.844955Z",
"shell.execute_reply": "2023-08-18T07:18:49.843595Z"
},
"id": "82e98083",
"origin_pos": 2,
"tab": [
"pytorch"
]
},
"outputs": [],
"source": [
"import torch\n",
"from torch import nn\n",
"from torch.nn import functional as F\n",
"from d2l import torch as d2l\n",
"\n",
"\n",
"class Inception(nn.Module):\n",
" # c1--c4是每条路径的输出通道数\n",
" def __init__(self, in_channels, c1, c2, c3, c4, **kwargs):\n",
" super(Inception, self).__init__(**kwargs)\n",
" # 线路1，单1x1卷积层\n",
" self.p1_1 = nn.Conv2d(in_channels, c1, kernel_size=1)\n",
" # 线路2，1x1卷积层后接3x3卷积层\n",
" self.p2_1 = nn.Conv2d(in_channels, c2[0], kernel_size=1)\n",
" self.p2_2 = nn.Conv2d(c2[0], c2[1], kernel_size=3, padding=1)\n",
" # 线路3，1x1卷积层后接5x5卷积层\n",
" self.p3_1 = nn.Conv2d(in_channels, c3[0], kernel_size=1)\n",
" self.p3_2 = nn.Conv2d(c3[0], c3[1], kernel_size=5, padding=2)\n",
" # 线路4，3x3最大汇聚层后接1x1卷积层\n",
" self.p4_1 = nn.MaxPool2d(kernel_size=3, stride=1, padding=1)\n",
" self.p4_2 = nn.Conv2d(in_channels, c4, kernel_size=1)\n",
"\n",
" def forward(self, x):\n",
" p1 = F.relu(self.p1_1(x))\n",
" p2 = F.relu(self.p2_2(F.relu(self.p2_1(x))))\n",
" p3 = F.relu(self.p3_2(F.relu(self.p3_1(x))))\n",
" p4 = F.relu(self.p4_2(self.p4_1(x)))\n",
" # 在通道维度上连结输出\n",
" return torch.cat((p1, p2, p3, p4), dim=1)"
]
},
{
"cell_type": "markdown",
"id": "67ed1e2a",
"metadata": {
"id": "67ed1e2a",
"origin_pos": 5
},
"source": [
"现在，我们逐一实现GoogLeNet的每个模块。第一个模块使用64个通道、$7\\times 7$卷积层。"
]
},
{
"cell_type": "code",
"execution_count": 11,
"id": "4278ba0a",
"metadata": {
"execution": {
"iopub.execute_input": "2023-08-18T07:18:49.850742Z",
"iopub.status.busy": "2023-08-18T07:18:49.849860Z",
"iopub.status.idle": "2023-08-18T07:18:49.877802Z",
"shell.execute_reply": "2023-08-18T07:18:49.876517Z"
},
"id": "4278ba0a",
"origin_pos": 7,
"tab": [
"pytorch"
]
},
"outputs": [],
"source": [
"b1 = nn.Sequential(nn.Conv2d(1, 64, kernel_size=7, stride=2, padding=3),\n",
" nn.ReLU(),\n",
" nn.MaxPool2d(kernel_size=3, stride=2, padding=1))"
]
},
{
"cell_type": "markdown",
"id": "b4d6d53a",
"metadata": {
"id": "b4d6d53a",
"origin_pos": 10
},
"source": [
"第二个模块使用两个卷积层：第一个卷积层是64个通道、$1\\times 1$卷积层；第二个卷积层使用将通道数量增加三倍的$3\\times 3$卷积层。\n",
"这对应于Inception块中的第二条路径。\n"
]
},
{
"cell_type": "code",
"execution_count": 12,
"id": "d5a1d1b0",
"metadata": {
"execution": {
"iopub.execute_input": "2023-08-18T07:18:49.883499Z",
"iopub.status.busy": "2023-08-18T07:18:49.882740Z",
"iopub.status.idle": "2023-08-18T07:18:49.892430Z",
"shell.execute_reply": "2023-08-18T07:18:49.891290Z"
},
"id": "d5a1d1b0",
"origin_pos": 12,
"tab": [
"pytorch"
]
},
"outputs": [],
"source": [
"b2 = nn.Sequential(nn.Conv2d(64, 64, kernel_size=1),\n",
" nn.ReLU(),\n",
" nn.Conv2d(64, 192, kernel_size=3, padding=1),\n",
" nn.ReLU(),\n",
" nn.MaxPool2d(kernel_size=3, stride=2, padding=1))"
]
},
{
"cell_type": "markdown",
"id": "c5532216",
"metadata": {
"id": "c5532216",
"origin_pos": 15
},
"source": [
"第三个模块串联两个完整的Inception块。\n",
"第一个Inception块的输出通道数为$64+128+32+32=256$，四个路径之间的输出通道数量比为$64:128:32:32=2:4:1:1$。\n",
"第二个和第三个路径首先将输入通道的数量分别减少到$96/192=1/2$和$16/192=1/12$，然后连接第二个卷积层。第二个Inception块的输出通道数增加到$128+192+96+64=480$，四个路径之间的输出通道数量比为$128:192:96:64 = 4:6:3:2$。\n",
"第二条和第三条路径首先将输入通道的数量分别减少到$128/256=1/2$和$32/256=1/8$。\n"
]
},
{
"cell_type": "code",
"execution_count": 13,
"id": "11b7cf54",
"metadata": {
"execution": {
"iopub.execute_input": "2023-08-18T07:18:49.897526Z",
"iopub.status.busy": "2023-08-18T07:18:49.896704Z",
"iopub.status.idle": "2023-08-18T07:18:49.916207Z",
"shell.execute_reply": "2023-08-18T07:18:49.915075Z"
},
"id": "11b7cf54",
"origin_pos": 17,
"tab": [
"pytorch"
]
},
"outputs": [],
"source": [
"b3 = nn.Sequential(Inception(192, 64, (96, 128), (16, 32), 32),\n",
" Inception(256, 128, (128, 192), (32, 96), 64),\n",
" nn.MaxPool2d(kernel_size=3, stride=2, padding=1))"
]
},
{
"cell_type": "markdown",
"id": "f54c6828",
"metadata": {
"id": "f54c6828",
"origin_pos": 20
},
"source": [
"第四模块更加复杂，\n",
"它串联了5个Inception块，其输出通道数分别是$192+208+48+64=512$、$160+224+64+64=512$、$128+256+64+64=512$、$112+288+64+64=528$和$256+320+128+128=832$。\n",
"这些路径的通道数分配和第三模块中的类似，首先是含$3×3$卷积层的第二条路径输出最多通道，其次是仅含$1×1$卷积层的第一条路径，之后是含$5×5$卷积层的第三条路径和含$3×3$最大汇聚层的第四条路径。\n",
"其中第二、第三条路径都会先按比例减小通道数。\n",
"这些比例在各个Inception块中都略有不同。\n"
]
},
{
"cell_type": "code",
"execution_count": 14,
"id": "b55bd896",
"metadata": {
"execution": {
"iopub.execute_input": "2023-08-18T07:18:49.921459Z",
"iopub.status.busy": "2023-08-18T07:18:49.920668Z",
"iopub.status.idle": "2023-08-18T07:18:49.984376Z",
"shell.execute_reply": "2023-08-18T07:18:49.982781Z"
},
"id": "b55bd896",
"origin_pos": 22,
"tab": [
"pytorch"
]
},
"outputs": [],
"source": [
"b4 = nn.Sequential(Inception(480, 192, (96, 208), (16, 48), 64),\n",
" Inception(512, 160, (112, 224), (24, 64), 64),\n",
" Inception(512, 128, (128, 256), (24, 64), 64),\n",
" Inception(512, 112, (144, 288), (32, 64), 64),\n",
" Inception(528, 256, (160, 320), (32, 128), 128),\n",
" nn.MaxPool2d(kernel_size=3, stride=2, padding=1))"
]
},
{
"cell_type": "markdown",
"id": "2a44e596",
"metadata": {
"id": "2a44e596",
"origin_pos": 25
},
"source": [
"第五模块包含输出通道数为$256+320+128+128=832$和$384+384+128+128=1024$的两个Inception块。\n",
"其中每条路径通道数的分配思路和第三、第四模块中的一致，只是在具体数值上有所不同。\n",
"需要注意的是，第五模块的后面紧跟输出层，该模块同NiN一样使用全局平均汇聚层，将每个通道的高和宽变成1。\n",
"最后我们将输出变成二维数组，再接上一个输出个数为标签类别数的全连接层。\n"
]
},
{
"cell_type": "code",
"execution_count": 15,
"id": "8fc4cc33",
"metadata": {
"execution": {
"iopub.execute_input": "2023-08-18T07:18:49.990643Z",
"iopub.status.busy": "2023-08-18T07:18:49.989777Z",
"iopub.status.idle": "2023-08-18T07:18:50.042949Z",
"shell.execute_reply": "2023-08-18T07:18:50.041737Z"
},
"id": "8fc4cc33",
"origin_pos": 27,
"tab": [
"pytorch"
]
},
"outputs": [],
"source": [
"b5 = nn.Sequential(Inception(832, 256, (160, 320), (32, 128), 128),\n",
" Inception(832, 384, (192, 384), (48, 128), 128),\n",
" nn.AdaptiveAvgPool2d((1,1)),\n",
" nn.Flatten())\n",
"\n",
"net = nn.Sequential(b1, b2, b3, b4, b5, nn.Linear(1024, 10))"
]
},
{
"cell_type": "markdown",
"id": "7e92ad19",
"metadata": {
"id": "7e92ad19",
"origin_pos": 30
},
"source": [
"GoogLeNet模型的计算复杂，而且不如VGG那样便于修改通道数。\n",
"[**为了使Fashion-MNIST上的训练短小精悍，我们将输入的高和宽从224降到96**]，这简化了计算。下面演示各个模块输出的形状变化。\n"
]
},
{
"cell_type": "code",
"execution_count": 16,
"id": "51845ad5",
"metadata": {
"execution": {
"iopub.execute_input": "2023-08-18T07:18:50.048543Z",
"iopub.status.busy": "2023-08-18T07:18:50.047764Z",
"iopub.status.idle": "2023-08-18T07:18:50.254447Z",
"shell.execute_reply": "2023-08-18T07:18:50.252245Z"
},
"id": "51845ad5",
"origin_pos": 32,
"outputId": "af2d6ac5-f803-413f-ea22-c148c50d8cac",
"tab": [
"pytorch"
]
},
"outputs": [
{
"name": "stdout",
"output_type": "stream",
"text": [
"Sequential output shape:\t torch.Size([1, 64, 24, 24])\n",
"Sequential output shape:\t torch.Size([1, 192, 12, 12])\n",
"Sequential output shape:\t torch.Size([1, 480, 6, 6])\n",
"Sequential output shape:\t torch.Size([1, 832, 3, 3])\n",
"Sequential output shape:\t torch.Size([1, 1024])\n",
"Linear output shape:\t torch.Size([1, 10])\n"
]
}
],
"source": [
"X = torch.rand(size=(1, 1, 96, 96))\n",
"for layer in net:\n",
" X = layer(X)\n",
" print(layer.__class__.__name__,'output shape:\\t', X.shape)"
]
},
{
"cell_type": "markdown",
"id": "1dfcf730",
"metadata": {
"id": "1dfcf730",
"origin_pos": 35
},
"source": [
"## [**训练模型**]\n",
"\n",
"和以前一样，我们使用Fashion-MNIST数据集来训练我们的模型。在训练之前，我们将图片转换为$96 \\times 96$分辨率。\n"
]
},
{
"cell_type": "code",
"execution_count": 17,
"id": "fe051d5b",
"metadata": {
"execution": {
"iopub.execute_input": "2023-08-18T07:18:50.260612Z",
"iopub.status.busy": "2023-08-18T07:18:50.259996Z",
"iopub.status.idle": "2023-08-18T07:22:36.284598Z",
"shell.execute_reply": "2023-08-18T07:22:36.283681Z"
},
"id": "fe051d5b",
"origin_pos": 36,
"outputId": "27c16169-64f0-472f-fb3f-206ca1afcbc8",
"tab": [
"pytorch"
]
},
"outputs": [
{
"name": "stdout",
"output_type": "stream",
"text": [
"loss 0.251, train acc 0.904, test acc 0.878\n",
"779.9 examples/sec on cuda:0\n"
]
},
{
"data": {
"image/svg+xml": [
"<?xml version=\"1.0\" encoding=\"utf-8\" standalone=\"no\"?>\n",
"<!DOCTYPE svg PUBLIC \"-//W3C//DTD SVG 1.1//EN\"\n",
" \"http://www.w3.org/Graphics/SVG/1.1/DTD/svg11.dtd\">\n",
"<svg xmlns:xlink=\"http://www.w3.org/1999/xlink\" width=\"238.965625pt\" height=\"183.35625pt\" viewBox=\"0 0 238.965625 183.35625\" xmlns=\"http://www.w3.org/2000/svg\" version=\"1.1\">\n",
" <metadata>\n",
" <rdf:RDF xmlns:dc=\"http://purl.org/dc/elements/1.1/\" xmlns:cc=\"http://creativecommons.org/ns#\" xmlns:rdf=\"http://www.w3.org/1999/02/22-rdf-syntax-ns#\">\n",
" <cc:Work>\n",
" <dc:type rdf:resource=\"http://purl.org/dc/dcmitype/StillImage\"/>\n",
" <dc:date>2023-11-14T19:29:55.138974</dc:date>\n",
" <dc:format>image/svg+xml</dc:format>\n",
" <dc:creator>\n",
" <cc:Agent>\n",
" <dc:title>Matplotlib v3.5.1, https://matplotlib.org/</dc:title>\n",
" </cc:Agent>\n",
" </dc:creator>\n",
" </cc:Work>\n",
" </rdf:RDF>\n",
" </metadata>\n",
" <defs>\n",
" <style type=\"text/css\">*{stroke-linejoin: round; stroke-linecap: butt}</style>\n",
" </defs>\n",
" <g id=\"figure_1\">\n",
" <g id=\"patch_1\">\n",
" <path d=\"M 0 183.35625 \n",
"L 238.965625 183.35625 \n",
"L 238.965625 0 \n",
"L 0 0 \n",
"z\n",
"\" style=\"fill: #ffffff\"/>\n",
" </g>\n",
" <g id=\"axes_1\">\n",
" <g id=\"patch_2\">\n",
" <path d=\"M 30.103125 145.8 \n",
"L 225.403125 145.8 \n",
"L 225.403125 7.2 \n",
"L 30.103125 7.2 \n",
"z\n",
"\" style=\"fill: #ffffff\"/>\n",
" </g>\n",
" <g id=\"matplotlib.axis_1\">\n",
" <g id=\"xtick_1\">\n",
" <g id=\"line2d_1\">\n",
" <path d=\"M 51.803125 145.8 \n",
"L 51.803125 7.2 \n",
"\" clip-path=\"url(#p5475e26869)\" style=\"fill: none; stroke: #b0b0b0; stroke-width: 0.8; stroke-linecap: square\"/>\n",
" </g>\n",
" <g id=\"line2d_2\">\n",
" <defs>\n",
" <path id=\"m87e4dd0f28\" d=\"M 0 0 \n",
"L 0 3.5 \n",
"\" style=\"stroke: #000000; stroke-width: 0.8\"/>\n",
" </defs>\n",
" <g>\n",
" <use xlink:href=\"#m87e4dd0f28\" x=\"51.803125\" y=\"145.8\" style=\"stroke: #000000; stroke-width: 0.8\"/>\n",
" </g>\n",
" </g>\n",
" <g id=\"text_1\">\n",
" <!-- 2 -->\n",
" <g transform=\"translate(48.621875 160.398438)scale(0.1 -0.1)\">\n",
" <defs>\n",
" <path id=\"DejaVuSans-32\" d=\"M 1228 531 \n",
"L 3431 531 \n",
"L 3431 0 \n",
"L 469 0 \n",
"L 469 531 \n",
"Q 828 903 1448 1529 \n",
"Q 2069 2156 2228 2338 \n",
"Q 2531 2678 2651 2914 \n",
"Q 2772 3150 2772 3378 \n",
"Q 2772 3750 2511 3984 \n",
"Q 2250 4219 1831 4219 \n",
"Q 1534 4219 1204 4116 \n",
"Q 875 4013 500 3803 \n",
"L 500 4441 \n",
"Q 881 4594 1212 4672 \n",
"Q 1544 4750 1819 4750 \n",
"Q 2544 4750 2975 4387 \n",
"Q 3406 4025 3406 3419 \n",
"Q 3406 3131 3298 2873 \n",
"Q 3191 2616 2906 2266 \n",
"Q 2828 2175 2409 1742 \n",
"Q 1991 1309 1228 531 \n",
"z\n",
"\" transform=\"scale(0.015625)\"/>\n",
" </defs>\n",
" <use xlink:href=\"#DejaVuSans-32\"/>\n",
" </g>\n",
" </g>\n",
" </g>\n",
" <g id=\"xtick_2\">\n",
" <g id=\"line2d_3\">\n",
" <path d=\"M 95.203125 145.8 \n",
"L 95.203125 7.2 \n",
"\" clip-path=\"url(#p5475e26869)\" style=\"fill: none; stroke: #b0b0b0; stroke-width: 0.8; stroke-linecap: square\"/>\n",
" </g>\n",
" <g id=\"line2d_4\">\n",
" <g>\n",
" <use xlink:href=\"#m87e4dd0f28\" x=\"95.203125\" y=\"145.8\" style=\"stroke: #000000; stroke-width: 0.8\"/>\n",
" </g>\n",
" </g>\n",
" <g id=\"text_2\">\n",
" <!-- 4 -->\n",
" <g transform=\"translate(92.021875 160.398438)scale(0.1 -0.1)\">\n",
" <defs>\n",
" <path id=\"DejaVuSans-34\" d=\"M 2419 4116 \n",
"L 825 1625 \n",
"L 2419 1625 \n",
"L 2419 4116 \n",
"z\n",
"M 2253 4666 \n",
"L 3047 4666 \n",
"L 3047 1625 \n",
"L 3713 1625 \n",
"L 3713 1100 \n",
"L 3047 1100 \n",
"L 3047 0 \n",
"L 2419 0 \n",
"L 2419 1100 \n",
"L 313 1100 \n",
"L 313 1709 \n",
"L 2253 4666 \n",
"z\n",
"\" transform=\"scale(0.015625)\"/>\n",
" </defs>\n",
" <use xlink:href=\"#DejaVuSans-34\"/>\n",
" </g>\n",
" </g>\n",
" </g>\n",
" <g id=\"xtick_3\">\n",
" <g id=\"line2d_5\">\n",
" <path d=\"M 138.603125 145.8 \n",
"L 138.603125 7.2 \n",
"\" clip-path=\"url(#p5475e26869)\" style=\"fill: none; stroke: #b0b0b0; stroke-width: 0.8; stroke-linecap: square\"/>\n",
" </g>\n",
" <g id=\"line2d_6\">\n",
" <g>\n",
" <use xlink:href=\"#m87e4dd0f28\" x=\"138.603125\" y=\"145.8\" style=\"stroke: #000000; stroke-width: 0.8\"/>\n",
" </g>\n",
" </g>\n",
" <g id=\"text_3\">\n",
" <!-- 6 -->\n",
" <g transform=\"translate(135.421875 160.398438)scale(0.1 -0.1)\">\n",
" <defs>\n",
" <path id=\"DejaVuSans-36\" d=\"M 2113 2584 \n",
"Q 1688 2584 1439 2293 \n",
"Q 1191 2003 1191 1497 \n",
"Q 1191 994 1439 701 \n",
"Q 1688 409 2113 409 \n",
"Q 2538 409 2786 701 \n",
"Q 3034 994 3034 1497 \n",
"Q 3034 2003 2786 2293 \n",
"Q 2538 2584 2113 2584 \n",
"z\n",
"M 3366 4563 \n",
"L 3366 3988 \n",
"Q 3128 4100 2886 4159 \n",
"Q 2644 4219 2406 4219 \n",
"Q 1781 4219 1451 3797 \n",
"Q 1122 3375 1075 2522 \n",
"Q 1259 2794 1537 2939 \n",
"Q 1816 3084 2150 3084 \n",
"Q 2853 3084 3261 2657 \n",
"Q 3669 2231 3669 1497 \n",
"Q 3669 778 3244 343 \n",
"Q 2819 -91 2113 -91 \n",
"Q 1303 -91 875 529 \n",
"Q 447 1150 447 2328 \n",
"Q 447 3434 972 4092 \n",
"Q 1497 4750 2381 4750 \n",
"Q 2619 4750 2861 4703 \n",
"Q 3103 4656 3366 4563 \n",
"z\n",
"\" transform=\"scale(0.015625)\"/>\n",
" </defs>\n",
" <use xlink:href=\"#DejaVuSans-36\"/>\n",
" </g>\n",
" </g>\n",
" </g>\n",
" <g id=\"xtick_4\">\n",
" <g id=\"line2d_7\">\n",
" <path d=\"M 182.003125 145.8 \n",
"L 182.003125 7.2 \n",
"\" clip-path=\"url(#p5475e26869)\" style=\"fill: none; stroke: #b0b0b0; stroke-width: 0.8; stroke-linecap: square\"/>\n",
" </g>\n",
" <g id=\"line2d_8\">\n",
" <g>\n",
" <use xlink:href=\"#m87e4dd0f28\" x=\"182.003125\" y=\"145.8\" style=\"stroke: #000000; stroke-width: 0.8\"/>\n",
" </g>\n",
" </g>\n",
" <g id=\"text_4\">\n",
" <!-- 8 -->\n",
" <g transform=\"translate(178.821875 160.398438)scale(0.1 -0.1)\">\n",
" <defs>\n",
" <path id=\"DejaVuSans-38\" d=\"M 2034 2216 \n",
"Q 1584 2216 1326 1975 \n",
"Q 1069 1734 1069 1313 \n",
"Q 1069 891 1326 650 \n",
"Q 1584 409 2034 409 \n",
"Q 2484 409 2743 651 \n",
"Q 3003 894 3003 1313 \n",
"Q 3003 1734 2745 1975 \n",
"Q 2488 2216 2034 2216 \n",
"z\n",
"M 1403 2484 \n",
"Q 997 2584 770 2862 \n",
"Q 544 3141 544 3541 \n",
"Q 544 4100 942 4425 \n",
"Q 1341 4750 2034 4750 \n",
"Q 2731 4750 3128 4425 \n",
"Q 3525 4100 3525 3541 \n",
"Q 3525 3141 3298 2862 \n",
"Q 3072 2584 2669 2484 \n",
"Q 3125 2378 3379 2068 \n",
"Q 3634 1759 3634 1313 \n",
"Q 3634 634 3220 271 \n",
"Q 2806 -91 2034 -91 \n",
"Q 1263 -91 848 271 \n",
"Q 434 634 434 1313 \n",
"Q 434 1759 690 2068 \n",
"Q 947 2378 1403 2484 \n",
"z\n",
"M 1172 3481 \n",
"Q 1172 3119 1398 2916 \n",
"Q 1625 2713 2034 2713 \n",
"Q 2441 2713 2670 2916 \n",
"Q 2900 3119 2900 3481 \n",
"Q 2900 3844 2670 4047 \n",
"Q 2441 4250 2034 4250 \n",
"Q 1625 4250 1398 4047 \n",
"Q 1172 3844 1172 3481 \n",
"z\n",
"\" transform=\"scale(0.015625)\"/>\n",
" </defs>\n",
" <use xlink:href=\"#DejaVuSans-38\"/>\n",
" </g>\n",
" </g>\n",
" </g>\n",
" <g id=\"xtick_5\">\n",
" <g id=\"line2d_9\">\n",
" <path d=\"M 225.403125 145.8 \n",
"L 225.403125 7.2 \n",
"\" clip-path=\"url(#p5475e26869)\" style=\"fill: none; stroke: #b0b0b0; stroke-width: 0.8; stroke-linecap: square\"/>\n",
" </g>\n",
" <g id=\"line2d_10\">\n",
" <g>\n",
" <use xlink:href=\"#m87e4dd0f28\" x=\"225.403125\" y=\"145.8\" style=\"stroke: #000000; stroke-width: 0.8\"/>\n",
" </g>\n",
" </g>\n",
" <g id=\"text_5\">\n",
" <!-- 10 -->\n",
" <g transform=\"translate(219.040625 160.398438)scale(0.1 -0.1)\">\n",
" <defs>\n",
" <path id=\"DejaVuSans-31\" d=\"M 794 531 \n",
"L 1825 531 \n",
"L 1825 4091 \n",
"L 703 3866 \n",
"L 703 4441 \n",
"L 1819 4666 \n",
"L 2450 4666 \n",
"L 2450 531 \n",
"L 3481 531 \n",
"L 3481 0 \n",
"L 794 0 \n",
"L 794 531 \n",
"z\n",
"\" transform=\"scale(0.015625)\"/>\n",
" <path id=\"DejaVuSans-30\" d=\"M 2034 4250 \n",
"Q 1547 4250 1301 3770 \n",
"Q 1056 3291 1056 2328 \n",
"Q 1056 1369 1301 889 \n",
"Q 1547 409 2034 409 \n",
"Q 2525 409 2770 889 \n",
"Q 3016 1369 3016 2328 \n",
"Q 3016 3291 2770 3770 \n",
"Q 2525 4250 2034 4250 \n",
"z\n",
"M 2034 4750 \n",
"Q 2819 4750 3233 4129 \n",
"Q 3647 3509 3647 2328 \n",
"Q 3647 1150 3233 529 \n",
"Q 2819 -91 2034 -91 \n",
"Q 1250 -91 836 529 \n",
"Q 422 1150 422 2328 \n",
"Q 422 3509 836 4129 \n",
"Q 1250 4750 2034 4750 \n",
"z\n",
"\" transform=\"scale(0.015625)\"/>\n",
" </defs>\n",
" <use xlink:href=\"#DejaVuSans-31\"/>\n",
" <use xlink:href=\"#DejaVuSans-30\" x=\"63.623047\"/>\n",
" </g>\n",
" </g>\n",
" </g>\n",
" <g id=\"text_6\">\n",
" <!-- epoch -->\n",
" <g transform=\"translate(112.525 174.076563)scale(0.1 -0.1)\">\n",
" <defs>\n",
" <path id=\"DejaVuSans-65\" d=\"M 3597 1894 \n",
"L 3597 1613 \n",
"L 953 1613 \n",
"Q 991 1019 1311 708 \n",
"Q 1631 397 2203 397 \n",
"Q 2534 397 2845 478 \n",
"Q 3156 559 3463 722 \n",
"L 3463 178 \n",
"Q 3153 47 2828 -22 \n",
"Q 2503 -91 2169 -91 \n",
"Q 1331 -91 842 396 \n",
"Q 353 884 353 1716 \n",
"Q 353 2575 817 3079 \n",
"Q 1281 3584 2069 3584 \n",
"Q 2775 3584 3186 3129 \n",
"Q 3597 2675 3597 1894 \n",
"z\n",
"M 3022 2063 \n",
"Q 3016 2534 2758 2815 \n",
"Q 2500 3097 2075 3097 \n",
"Q 1594 3097 1305 2825 \n",
"Q 1016 2553 972 2059 \n",
"L 3022 2063 \n",
"z\n",
"\" transform=\"scale(0.015625)\"/>\n",
" <path id=\"DejaVuSans-70\" d=\"M 1159 525 \n",
"L 1159 -1331 \n",
"L 581 -1331 \n",
"L 581 3500 \n",
"L 1159 3500 \n",
"L 1159 2969 \n",
"Q 1341 3281 1617 3432 \n",
"Q 1894 3584 2278 3584 \n",
"Q 2916 3584 3314 3078 \n",
"Q 3713 2572 3713 1747 \n",
"Q 3713 922 3314 415 \n",
"Q 2916 -91 2278 -91 \n",
"Q 1894 -91 1617 61 \n",
"Q 1341 213 1159 525 \n",
"z\n",
"M 3116 1747 \n",
"Q 3116 2381 2855 2742 \n",
"Q 2594 3103 2138 3103 \n",
"Q 1681 3103 1420 2742 \n",
"Q 1159 2381 1159 1747 \n",
"Q 1159 1113 1420 752 \n",
"Q 1681 391 2138 391 \n",
"Q 2594 391 2855 752 \n",
"Q 3116 1113 3116 1747 \n",
"z\n",
"\" transform=\"scale(0.015625)\"/>\n",
" <path id=\"DejaVuSans-6f\" d=\"M 1959 3097 \n",
"Q 1497 3097 1228 2736 \n",
"Q 959 2375 959 1747 \n",
"Q 959 1119 1226 758 \n",
"Q 1494 397 1959 397 \n",
"Q 2419 397 2687 759 \n",
"Q 2956 1122 2956 1747 \n",
"Q 2956 2369 2687 2733 \n",
"Q 2419 3097 1959 3097 \n",
"z\n",
"M 1959 3584 \n",
"Q 2709 3584 3137 3096 \n",
"Q 3566 2609 3566 1747 \n",
"Q 3566 888 3137 398 \n",
"Q 2709 -91 1959 -91 \n",
"Q 1206 -91 779 398 \n",
"Q 353 888 353 1747 \n",
"Q 353 2609 779 3096 \n",
"Q 1206 3584 1959 3584 \n",
"z\n",
"\" transform=\"scale(0.015625)\"/>\n",
" <path id=\"DejaVuSans-63\" d=\"M 3122 3366 \n",
"L 3122 2828 \n",
"Q 2878 2963 2633 3030 \n",
"Q 2388 3097 2138 3097 \n",
"Q 1578 3097 1268 2742 \n",
"Q 959 2388 959 1747 \n",
"Q 959 1106 1268 751 \n",
"Q 1578 397 2138 397 \n",
"Q 2388 397 2633 464 \n",
"Q 2878 531 3122 666 \n",
"L 3122 134 \n",
"Q 2881 22 2623 -34 \n",
"Q 2366 -91 2075 -91 \n",
"Q 1284 -91 818 406 \n",
"Q 353 903 353 1747 \n",
"Q 353 2603 823 3093 \n",
"Q 1294 3584 2113 3584 \n",
"Q 2378 3584 2631 3529 \n",
"Q 2884 3475 3122 3366 \n",
"z\n",
"\" transform=\"scale(0.015625)\"/>\n",
" <path id=\"DejaVuSans-68\" d=\"M 3513 2113 \n",
"L 3513 0 \n",
"L 2938 0 \n",
"L 2938 2094 \n",
"Q 2938 2591 2744 2837 \n",
"Q 2550 3084 2163 3084 \n",
"Q 1697 3084 1428 2787 \n",
"Q 1159 2491 1159 1978 \n",
"L 1159 0 \n",
"L 581 0 \n",
"L 581 4863 \n",
"L 1159 4863 \n",
"L 1159 2956 \n",
"Q 1366 3272 1645 3428 \n",
"Q 1925 3584 2291 3584 \n",
"Q 2894 3584 3203 3211 \n",
"Q 3513 2838 3513 2113 \n",
"z\n",
"\" transform=\"scale(0.015625)\"/>\n",
" </defs>\n",
" <use xlink:href=\"#DejaVuSans-65\"/>\n",
" <use xlink:href=\"#DejaVuSans-70\" x=\"61.523438\"/>\n",
" <use xlink:href=\"#DejaVuSans-6f\" x=\"125\"/>\n",
" <use xlink:href=\"#DejaVuSans-63\" x=\"186.181641\"/>\n",
" <use xlink:href=\"#DejaVuSans-68\" x=\"241.162109\"/>\n",
" </g>\n",
" </g>\n",
" </g>\n",
" <g id=\"matplotlib.axis_2\">\n",
" <g id=\"ytick_1\">\n",
" <g id=\"line2d_11\">\n",
" <path d=\"M 30.103125 145.754944 \n",
"L 225.403125 145.754944 \n",
"\" clip-path=\"url(#p5475e26869)\" style=\"fill: none; stroke: #b0b0b0; stroke-width: 0.8; stroke-linecap: square\"/>\n",
" </g>\n",
" <g id=\"line2d_12\">\n",
" <defs>\n",
" <path id=\"m311983c4fc\" d=\"M 0 0 \n",
"L -3.5 0 \n",
"\" style=\"stroke: #000000; stroke-width: 0.8\"/>\n",
" </defs>\n",
" <g>\n",
" <use xlink:href=\"#m311983c4fc\" x=\"30.103125\" y=\"145.754944\" style=\"stroke: #000000; stroke-width: 0.8\"/>\n",
" </g>\n",
" </g>\n",
" <g id=\"text_7\">\n",
" <!-- 0.0 -->\n",
" <g transform=\"translate(7.2 149.554163)scale(0.1 -0.1)\">\n",
" <defs>\n",
" <path id=\"DejaVuSans-2e\" d=\"M 684 794 \n",
"L 1344 794 \n",
"L 1344 0 \n",
"L 684 0 \n",
"L 684 794 \n",
"z\n",
"\" transform=\"scale(0.015625)\"/>\n",
" </defs>\n",
" <use xlink:href=\"#DejaVuSans-30\"/>\n",
" <use xlink:href=\"#DejaVuSans-2e\" x=\"63.623047\"/>\n",
" <use xlink:href=\"#DejaVuSans-30\" x=\"95.410156\"/>\n",
" </g>\n",
" </g>\n",
" </g>\n",
" <g id=\"ytick_2\">\n",
" <g id=\"line2d_13\">\n",
" <path d=\"M 30.103125 117.028534 \n",
"L 225.403125 117.028534 \n",
"\" clip-path=\"url(#p5475e26869)\" style=\"fill: none; stroke: #b0b0b0; stroke-width: 0.8; stroke-linecap: square\"/>\n",
" </g>\n",
" <g id=\"line2d_14\">\n",
" <g>\n",
" <use xlink:href=\"#m311983c4fc\" x=\"30.103125\" y=\"117.028534\" style=\"stroke: #000000; stroke-width: 0.8\"/>\n",
" </g>\n",
" </g>\n",
" <g id=\"text_8\">\n",
" <!-- 0.5 -->\n",
" <g transform=\"translate(7.2 120.827753)scale(0.1 -0.1)\">\n",
" <defs>\n",
" <path id=\"DejaVuSans-35\" d=\"M 691 4666 \n",
"L 3169 4666 \n",
"L 3169 4134 \n",
"L 1269 4134 \n",
"L 1269 2991 \n",
"Q 1406 3038 1543 3061 \n",
"Q 1681 3084 1819 3084 \n",
"Q 2600 3084 3056 2656 \n",
"Q 3513 2228 3513 1497 \n",
"Q 3513 744 3044 326 \n",
"Q 2575 -91 1722 -91 \n",
"Q 1428 -91 1123 -41 \n",
"Q 819 9 494 109 \n",
"L 494 744 \n",
"Q 775 591 1075 516 \n",
"Q 1375 441 1709 441 \n",
"Q 2250 441 2565 725 \n",
"Q 2881 1009 2881 1497 \n",
"Q 2881 1984 2565 2268 \n",
"Q 2250 2553 1709 2553 \n",
"Q 1456 2553 1204 2497 \n",
"Q 953 2441 691 2322 \n",
"L 691 4666 \n",
"z\n",
"\" transform=\"scale(0.015625)\"/>\n",
" </defs>\n",
" <use xlink:href=\"#DejaVuSans-30\"/>\n",
" <use xlink:href=\"#DejaVuSans-2e\" x=\"63.623047\"/>\n",
" <use xlink:href=\"#DejaVuSans-35\" x=\"95.410156\"/>\n",
" </g>\n",
" </g>\n",
" </g>\n",
" <g id=\"ytick_3\">\n",
" <g id=\"line2d_15\">\n",
" <path d=\"M 30.103125 88.302124 \n",
"L 225.403125 88.302124 \n",
"\" clip-path=\"url(#p5475e26869)\" style=\"fill: none; stroke: #b0b0b0; stroke-width: 0.8; stroke-linecap: square\"/>\n",
" </g>\n",
" <g id=\"line2d_16\">\n",
" <g>\n",
" <use xlink:href=\"#m311983c4fc\" x=\"30.103125\" y=\"88.302124\" style=\"stroke: #000000; stroke-width: 0.8\"/>\n",
" </g>\n",
" </g>\n",
" <g id=\"text_9\">\n",
" <!-- 1.0 -->\n",
" <g transform=\"translate(7.2 92.101343)scale(0.1 -0.1)\">\n",
" <use xlink:href=\"#DejaVuSans-31\"/>\n",
" <use xlink:href=\"#DejaVuSans-2e\" x=\"63.623047\"/>\n",
" <use xlink:href=\"#DejaVuSans-30\" x=\"95.410156\"/>\n",
" </g>\n",
" </g>\n",
" </g>\n",
" <g id=\"ytick_4\">\n",
" <g id=\"line2d_17\">\n",
" <path d=\"M 30.103125 59.575714 \n",
"L 225.403125 59.575714 \n",
"\" clip-path=\"url(#p5475e26869)\" style=\"fill: none; stroke: #b0b0b0; stroke-width: 0.8; stroke-linecap: square\"/>\n",
" </g>\n",
" <g id=\"line2d_18\">\n",
" <g>\n",
" <use xlink:href=\"#m311983c4fc\" x=\"30.103125\" y=\"59.575714\" style=\"stroke: #000000; stroke-width: 0.8\"/>\n",
" </g>\n",
" </g>\n",
" <g id=\"text_10\">\n",
" <!-- 1.5 -->\n",
" <g transform=\"translate(7.2 63.374933)scale(0.1 -0.1)\">\n",
" <use xlink:href=\"#DejaVuSans-31\"/>\n",
" <use xlink:href=\"#DejaVuSans-2e\" x=\"63.623047\"/>\n",
" <use xlink:href=\"#DejaVuSans-35\" x=\"95.410156\"/>\n",
" </g>\n",
" </g>\n",
" </g>\n",
" <g id=\"ytick_5\">\n",
" <g id=\"line2d_19\">\n",
" <path d=\"M 30.103125 30.849304 \n",
"L 225.403125 30.849304 \n",
"\" clip-path=\"url(#p5475e26869)\" style=\"fill: none; stroke: #b0b0b0; stroke-width: 0.8; stroke-linecap: square\"/>\n",
" </g>\n",
" <g id=\"line2d_20\">\n",
" <g>\n",
" <use xlink:href=\"#m311983c4fc\" x=\"30.103125\" y=\"30.849304\" style=\"stroke: #000000; stroke-width: 0.8\"/>\n",
" </g>\n",
" </g>\n",
" <g id=\"text_11\">\n",
" <!-- 2.0 -->\n",
" <g transform=\"translate(7.2 34.648523)scale(0.1 -0.1)\">\n",
" <use xlink:href=\"#DejaVuSans-32\"/>\n",
" <use xlink:href=\"#DejaVuSans-2e\" x=\"63.623047\"/>\n",
" <use xlink:href=\"#DejaVuSans-30\" x=\"95.410156\"/>\n",
" </g>\n",
" </g>\n",
" </g>\n",
" </g>\n",
" <g id=\"line2d_21\">\n",
" <path d=\"M 12.70611 13.5 \n",
"L 17.009095 13.733238 \n",
"L 21.31208 15.306866 \n",
"L 25.615065 23.26302 \n",
"L 29.91805 33.848416 \n",
"L 30.103125 34.304565 \n",
"L 34.40611 89.978335 \n",
"L 38.709095 93.012747 \n",
"L 43.01208 94.020927 \n",
"L 47.315065 96.69349 \n",
"L 51.61805 99.505013 \n",
"L 51.803125 99.633456 \n",
"L 56.10611 107.653638 \n",
"L 60.409095 111.654551 \n",
"L 64.71208 113.459415 \n",
"L 69.015065 114.591158 \n",
"L 73.31805 115.49524 \n",
"L 73.503125 115.543171 \n",
"L 77.80611 120.978491 \n",
"L 82.109095 121.523504 \n",
"L 86.41208 121.85909 \n",
"L 90.715065 121.996633 \n",
"L 95.01805 122.201079 \n",
"L 95.203125 122.221855 \n",
"L 99.50611 123.989483 \n",
"L 103.809095 124.249144 \n",
"L 108.11208 124.742427 \n",
"L 112.415065 125.109266 \n",
"L 116.71805 125.49245 \n",
"L 116.903125 125.522363 \n",
"L 121.20611 126.849843 \n",
"L 125.509095 127.122413 \n",
"L 129.81208 115.379794 \n",
"L 134.115065 117.78761 \n",
"L 138.41805 119.512887 \n",
"L 138.603125 119.555577 \n",
"L 142.90611 127.214383 \n",
"L 147.209095 127.338123 \n",
"L 151.51208 127.485925 \n",
"L 155.815065 127.773311 \n",
"L 160.11805 127.940813 \n",
"L 160.303125 127.915476 \n",
"L 164.60611 129.074678 \n",
"L 168.909095 129.454482 \n",
"L 173.21208 129.681102 \n",
"L 177.515065 129.419796 \n",
"L 181.81805 129.427145 \n",
"L 182.003125 129.455383 \n",
"L 186.30611 130.28108 \n",
"L 190.609095 130.321315 \n",
"L 194.91208 130.509806 \n",
"L 199.215065 130.367394 \n",
"L 203.51805 130.391588 \n",
"L 203.703125 130.40975 \n",
"L 208.00611 132.122729 \n",
"L 212.309095 131.506432 \n",
"L 216.61208 131.469197 \n",
"L 220.915065 131.474497 \n",
"L 225.21805 131.366628 \n",
"L 225.403125 131.347106 \n",
"\" clip-path=\"url(#p5475e26869)\" style=\"fill: none; stroke: #1f77b4; stroke-width: 1.5; stroke-linecap: square\"/>\n",
" </g>\n",
" <g id=\"line2d_22\">\n",
" <path d=\"M 12.70611 139.5 \n",
"L 17.009095 138.913599 \n",
"L 21.31208 137.731949 \n",
"L 25.615065 134.93307 \n",
"L 29.91805 131.428419 \n",
"L 30.103125 131.276833 \n",
"L 34.40611 111.72438 \n",
"L 38.709095 109.972417 \n",
"L 43.01208 109.230768 \n",
"L 47.315065 107.874163 \n",
"L 51.61805 106.600731 \n",
"L 51.803125 106.54531 \n",
"L 56.10611 102.250263 \n",
"L 60.409095 100.869928 \n",
"L 64.71208 100.310072 \n",
"L 69.015065 99.904659 \n",
"L 73.31805 99.623766 \n",
"L 73.503125 99.615542 \n",
"L 77.80611 97.588013 \n",
"L 82.109095 97.477007 \n",
"L 86.41208 97.390133 \n",
"L 90.715065 97.353936 \n",
"L 95.01805 97.249204 \n",
"L 95.203125 97.248486 \n",
"L 99.50611 96.603439 \n",
"L 103.809095 96.456235 \n",
"L 108.11208 96.289726 \n",
"L 112.415065 96.201645 \n",
"L 116.71805 96.067714 \n",
"L 116.903125 96.053467 \n",
"L 121.20611 95.618864 \n",
"L 125.509095 95.449942 \n",
"L 129.81208 99.246667 \n",
"L 134.115065 98.470028 \n",
"L 138.41805 97.903656 \n",
"L 138.603125 97.891 \n",
"L 142.90611 95.242409 \n",
"L 147.209095 95.278607 \n",
"L 151.51208 95.198972 \n",
"L 155.815065 95.130197 \n",
"L 160.11805 95.019432 \n",
"L 160.303125 95.028892 \n",
"L 164.60611 94.706685 \n",
"L 168.909095 94.523284 \n",
"L 173.21208 94.483064 \n",
"L 177.515065 94.552242 \n",
"L 181.81805 94.535832 \n",
"L 182.003125 94.53001 \n",
"L 186.30611 94.13235 \n",
"L 190.609095 94.192679 \n",
"L 194.91208 94.096957 \n",
"L 199.215065 94.115458 \n",
"L 203.51805 94.139107 \n",
"L 203.703125 94.132628 \n",
"L 208.00611 93.591799 \n",
"L 212.309095 93.72211 \n",
"L 216.61208 93.720502 \n",
"L 220.915065 93.720904 \n",
"L 225.21805 93.779061 \n",
"L 225.403125 93.788868 \n",
"\" clip-path=\"url(#p5475e26869)\" style=\"fill: none; stroke-dasharray: 5.55,2.4; stroke-dashoffset: 0; stroke: #bf00bf; stroke-width: 1.5\"/>\n",
" </g>\n",
" <g id=\"line2d_23\">\n",
" <path d=\"M 30.103125 115.557742 \n",
"L 51.803125 100.890037 \n",
"L 73.503125 98.132302 \n",
"L 95.203125 97.431377 \n",
"L 116.903125 95.828443 \n",
"L 138.603125 96.006547 \n",
"L 160.303125 95.55267 \n",
"L 182.003125 94.868981 \n",
"L 203.703125 94.242746 \n",
"L 225.403125 95.334349 \n",
"\" clip-path=\"url(#p5475e26869)\" style=\"fill: none; stroke-dasharray: 9.6,2.4,1.5,2.4; stroke-dashoffset: 0; stroke: #008000; stroke-width: 1.5\"/>\n",
" </g>\n",
" <g id=\"patch_3\">\n",
" <path d=\"M 30.103125 145.8 \n",
"L 30.103125 7.2 \n",
"\" style=\"fill: none; stroke: #000000; stroke-width: 0.8; stroke-linejoin: miter; stroke-linecap: square\"/>\n",
" </g>\n",
" <g id=\"patch_4\">\n",
" <path d=\"M 225.403125 145.8 \n",
"L 225.403125 7.2 \n",
"\" style=\"fill: none; stroke: #000000; stroke-width: 0.8; stroke-linejoin: miter; stroke-linecap: square\"/>\n",
" </g>\n",
" <g id=\"patch_5\">\n",
" <path d=\"M 30.103125 145.8 \n",
"L 225.403125 145.8 \n",
"\" style=\"fill: none; stroke: #000000; stroke-width: 0.8; stroke-linejoin: miter; stroke-linecap: square\"/>\n",
" </g>\n",
" <g id=\"patch_6\">\n",
" <path d=\"M 30.103125 7.2 \n",
"L 225.403125 7.2 \n",
"\" style=\"fill: none; stroke: #000000; stroke-width: 0.8; stroke-linejoin: miter; stroke-linecap: square\"/>\n",
" </g>\n",
" <g id=\"legend_1\">\n",
" <g id=\"patch_7\">\n",
" <path d=\"M 140.634375 59.234375 \n",
"L 218.403125 59.234375 \n",
"Q 220.403125 59.234375 220.403125 57.234375 \n",
"L 220.403125 14.2 \n",
"Q 220.403125 12.2 218.403125 12.2 \n",
"L 140.634375 12.2 \n",
"Q 138.634375 12.2 138.634375 14.2 \n",
"L 138.634375 57.234375 \n",
"Q 138.634375 59.234375 140.634375 59.234375 \n",
"z\n",
"\" style=\"fill: #ffffff; opacity: 0.8; stroke: #cccccc; stroke-linejoin: miter\"/>\n",
" </g>\n",
" <g id=\"line2d_24\">\n",
" <path d=\"M 142.634375 20.298438 \n",
"L 152.634375 20.298438 \n",
"L 162.634375 20.298438 \n",
"\" style=\"fill: none; stroke: #1f77b4; stroke-width: 1.5; stroke-linecap: square\"/>\n",
" </g>\n",
" <g id=\"text_12\">\n",
" <!-- train loss -->\n",
" <g transform=\"translate(170.634375 23.798438)scale(0.1 -0.1)\">\n",
" <defs>\n",
" <path id=\"DejaVuSans-74\" d=\"M 1172 4494 \n",
"L 1172 3500 \n",
"L 2356 3500 \n",
"L 2356 3053 \n",
"L 1172 3053 \n",
"L 1172 1153 \n",
"Q 1172 725 1289 603 \n",
"Q 1406 481 1766 481 \n",
"L 2356 481 \n",
"L 2356 0 \n",
"L 1766 0 \n",
"Q 1100 0 847 248 \n",
"Q 594 497 594 1153 \n",
"L 594 3053 \n",
"L 172 3053 \n",
"L 172 3500 \n",
"L 594 3500 \n",
"L 594 4494 \n",
"L 1172 4494 \n",
"z\n",
"\" transform=\"scale(0.015625)\"/>\n",
" <path id=\"DejaVuSans-72\" d=\"M 2631 2963 \n",
"Q 2534 3019 2420 3045 \n",
"Q 2306 3072 2169 3072 \n",
"Q 1681 3072 1420 2755 \n",
"Q 1159 2438 1159 1844 \n",
"L 1159 0 \n",
"L 581 0 \n",
"L 581 3500 \n",
"L 1159 3500 \n",
"L 1159 2956 \n",
"Q 1341 3275 1631 3429 \n",
"Q 1922 3584 2338 3584 \n",
"Q 2397 3584 2469 3576 \n",
"Q 2541 3569 2628 3553 \n",
"L 2631 2963 \n",
"z\n",
"\" transform=\"scale(0.015625)\"/>\n",
" <path id=\"DejaVuSans-61\" d=\"M 2194 1759 \n",
"Q 1497 1759 1228 1600 \n",
"Q 959 1441 959 1056 \n",
"Q 959 750 1161 570 \n",
"Q 1363 391 1709 391 \n",
"Q 2188 391 2477 730 \n",
"Q 2766 1069 2766 1631 \n",
"L 2766 1759 \n",
"L 2194 1759 \n",
"z\n",
"M 3341 1997 \n",
"L 3341 0 \n",
"L 2766 0 \n",
"L 2766 531 \n",
"Q 2569 213 2275 61 \n",
"Q 1981 -91 1556 -91 \n",
"Q 1019 -91 701 211 \n",
"Q 384 513 384 1019 \n",
"Q 384 1609 779 1909 \n",
"Q 1175 2209 1959 2209 \n",
"L 2766 2209 \n",
"L 2766 2266 \n",
"Q 2766 2663 2505 2880 \n",
"Q 2244 3097 1772 3097 \n",
"Q 1472 3097 1187 3025 \n",
"Q 903 2953 641 2809 \n",
"L 641 3341 \n",
"Q 956 3463 1253 3523 \n",
"Q 1550 3584 1831 3584 \n",
"Q 2591 3584 2966 3190 \n",
"Q 3341 2797 3341 1997 \n",
"z\n",
"\" transform=\"scale(0.015625)\"/>\n",
" <path id=\"DejaVuSans-69\" d=\"M 603 3500 \n",
"L 1178 3500 \n",
"L 1178 0 \n",
"L 603 0 \n",
"L 603 3500 \n",
"z\n",
"M 603 4863 \n",
"L 1178 4863 \n",
"L 1178 4134 \n",
"L 603 4134 \n",
"L 603 4863 \n",
"z\n",
"\" transform=\"scale(0.015625)\"/>\n",
" <path id=\"DejaVuSans-6e\" d=\"M 3513 2113 \n",
"L 3513 0 \n",
"L 2938 0 \n",
"L 2938 2094 \n",
"Q 2938 2591 2744 2837 \n",
"Q 2550 3084 2163 3084 \n",
"Q 1697 3084 1428 2787 \n",
"Q 1159 2491 1159 1978 \n",
"L 1159 0 \n",
"L 581 0 \n",
"L 581 3500 \n",
"L 1159 3500 \n",
"L 1159 2956 \n",
"Q 1366 3272 1645 3428 \n",
"Q 1925 3584 2291 3584 \n",
"Q 2894 3584 3203 3211 \n",
"Q 3513 2838 3513 2113 \n",
"z\n",
"\" transform=\"scale(0.015625)\"/>\n",
" <path id=\"DejaVuSans-20\" transform=\"scale(0.015625)\"/>\n",
" <path id=\"DejaVuSans-6c\" d=\"M 603 4863 \n",
"L 1178 4863 \n",
"L 1178 0 \n",
"L 603 0 \n",
"L 603 4863 \n",
"z\n",
"\" transform=\"scale(0.015625)\"/>\n",
" <path id=\"DejaVuSans-73\" d=\"M 2834 3397 \n",
"L 2834 2853 \n",
"Q 2591 2978 2328 3040 \n",
"Q 2066 3103 1784 3103 \n",
"Q 1356 3103 1142 2972 \n",
"Q 928 2841 928 2578 \n",
"Q 928 2378 1081 2264 \n",
"Q 1234 2150 1697 2047 \n",
"L 1894 2003 \n",
"Q 2506 1872 2764 1633 \n",
"Q 3022 1394 3022 966 \n",
"Q 3022 478 2636 193 \n",
"Q 2250 -91 1575 -91 \n",
"Q 1294 -91 989 -36 \n",
"Q 684 19 347 128 \n",
"L 347 722 \n",
"Q 666 556 975 473 \n",
"Q 1284 391 1588 391 \n",
"Q 1994 391 2212 530 \n",
"Q 2431 669 2431 922 \n",
"Q 2431 1156 2273 1281 \n",
"Q 2116 1406 1581 1522 \n",
"L 1381 1569 \n",
"Q 847 1681 609 1914 \n",
"Q 372 2147 372 2553 \n",
"Q 372 3047 722 3315 \n",
"Q 1072 3584 1716 3584 \n",
"Q 2034 3584 2315 3537 \n",
"Q 2597 3491 2834 3397 \n",
"z\n",
"\" transform=\"scale(0.015625)\"/>\n",
" </defs>\n",
" <use xlink:href=\"#DejaVuSans-74\"/>\n",
" <use xlink:href=\"#DejaVuSans-72\" x=\"39.208984\"/>\n",
" <use xlink:href=\"#DejaVuSans-61\" x=\"80.322266\"/>\n",
" <use xlink:href=\"#DejaVuSans-69\" x=\"141.601562\"/>\n",
" <use xlink:href=\"#DejaVuSans-6e\" x=\"169.384766\"/>\n",
" <use xlink:href=\"#DejaVuSans-20\" x=\"232.763672\"/>\n",
" <use xlink:href=\"#DejaVuSans-6c\" x=\"264.550781\"/>\n",
" <use xlink:href=\"#DejaVuSans-6f\" x=\"292.333984\"/>\n",
" <use xlink:href=\"#DejaVuSans-73\" x=\"353.515625\"/>\n",
" <use xlink:href=\"#DejaVuSans-73\" x=\"405.615234\"/>\n",
" </g>\n",
" </g>\n",
" <g id=\"line2d_25\">\n",
" <path d=\"M 142.634375 34.976563 \n",
"L 152.634375 34.976563 \n",
"L 162.634375 34.976563 \n",
"\" style=\"fill: none; stroke-dasharray: 5.55,2.4; stroke-dashoffset: 0; stroke: #bf00bf; stroke-width: 1.5\"/>\n",
" </g>\n",
" <g id=\"text_13\">\n",
" <!-- train acc -->\n",
" <g transform=\"translate(170.634375 38.476563)scale(0.1 -0.1)\">\n",
" <use xlink:href=\"#DejaVuSans-74\"/>\n",
" <use xlink:href=\"#DejaVuSans-72\" x=\"39.208984\"/>\n",
" <use xlink:href=\"#DejaVuSans-61\" x=\"80.322266\"/>\n",
" <use xlink:href=\"#DejaVuSans-69\" x=\"141.601562\"/>\n",
" <use xlink:href=\"#DejaVuSans-6e\" x=\"169.384766\"/>\n",
" <use xlink:href=\"#DejaVuSans-20\" x=\"232.763672\"/>\n",
" <use xlink:href=\"#DejaVuSans-61\" x=\"264.550781\"/>\n",
" <use xlink:href=\"#DejaVuSans-63\" x=\"325.830078\"/>\n",
" <use xlink:href=\"#DejaVuSans-63\" x=\"380.810547\"/>\n",
" </g>\n",
" </g>\n",
" <g id=\"line2d_26\">\n",
" <path d=\"M 142.634375 49.654688 \n",
"L 152.634375 49.654688 \n",
"L 162.634375 49.654688 \n",
"\" style=\"fill: none; stroke-dasharray: 9.6,2.4,1.5,2.4; stroke-dashoffset: 0; stroke: #008000; stroke-width: 1.5\"/>\n",
" </g>\n",
" <g id=\"text_14\">\n",
" <!-- test acc -->\n",
" <g transform=\"translate(170.634375 53.154688)scale(0.1 -0.1)\">\n",
" <use xlink:href=\"#DejaVuSans-74\"/>\n",
" <use xlink:href=\"#DejaVuSans-65\" x=\"39.208984\"/>\n",
" <use xlink:href=\"#DejaVuSans-73\" x=\"100.732422\"/>\n",
" <use xlink:href=\"#DejaVuSans-74\" x=\"152.832031\"/>\n",
" <use xlink:href=\"#DejaVuSans-20\" x=\"192.041016\"/>\n",
" <use xlink:href=\"#DejaVuSans-61\" x=\"223.828125\"/>\n",
" <use xlink:href=\"#DejaVuSans-63\" x=\"285.107422\"/>\n",
" <use xlink:href=\"#DejaVuSans-63\" x=\"340.087891\"/>\n",
" </g>\n",
" </g>\n",
" </g>\n",
" </g>\n",
" </g>\n",
" <defs>\n",
" <clipPath id=\"p5475e26869\">\n",
" <rect x=\"30.103125\" y=\"7.2\" width=\"195.3\" height=\"138.6\"/>\n",
" </clipPath>\n",
" </defs>\n",
"</svg>\n"
],
"text/plain": [
"<Figure size 350x250 with 1 Axes>"
]
},
"metadata": {},
"output_type": "display_data"
}
],
"source": [
"lr, num_epochs, batch_size = 0.1, 10, 128\n",
"train_iter, test_iter = d2l.load_data_fashion_mnist(batch_size, resize=96)\n",
"d2l.train_ch6(net, train_iter, test_iter, num_epochs, lr, d2l.try_gpu())"
]
},
{
"cell_type": "markdown",
"id": "535046e9",
"metadata": {
"id": "535046e9",
"origin_pos": 37
},
"source": [
"## 小结\n",
"\n",
"* Inception块相当于一个有4条路径的子网络。它通过不同窗口形状的卷积层和最大汇聚层来并行抽取信息，并使用$1×1$卷积层减少每像素级别上的通道维数从而降低模型复杂度。\n",
"* GoogLeNet将多个设计精细的Inception块与其他层（卷积层、全连接层）串联起来。其中Inception块的通道数分配之比是在ImageNet数据集上通过大量的实验得来的。\n",
"* GoogLeNet和它的后继者们一度是ImageNet上最有效的模型之一：它以较低的计算复杂度提供了类似的测试精度。\n",
"\n",
"## 练习\n",
"\n",
"1. GoogLeNet有一些后续版本。尝试实现并运行它们，然后观察实验结果。这些后续版本包括：\n",
" * 添加批量规范化层 :cite:`Ioffe.Szegedy.2015`（batch normalization），在 :numref:`sec_batch_norm`中将介绍；\n",
" * 对Inception模块进行调整 :cite:`Szegedy.Vanhoucke.Ioffe.ea.2016`；\n",
" * 使用标签平滑（label smoothing）进行模型正则化 :cite:`Szegedy.Vanhoucke.Ioffe.ea.2016`；\n",
" * 加入残差连接 :cite:`Szegedy.Ioffe.Vanhoucke.ea.2017`。（ :numref:`sec_resnet`将介绍）。\n",
"1. 使用GoogLeNet的最小图像大小是多少？\n",
"1. 将AlexNet、VGG和NiN的模型参数大小与GoogLeNet进行比较。后两个网络架构是如何显著减少模型参数大小的？\n"
]
},
{
"cell_type": "markdown",
"id": "cf7d384c",
"metadata": {
"id": "cf7d384c",
"origin_pos": 39,
"tab": [
"pytorch"
]
},
"source": [
"[Discussions](https://discuss.d2l.ai/t/1871)\n"
]
}
],
"metadata": {
"colab": {
"provenance": []
},
"kernelspec": {
"display_name": "Python 3 (ipykernel)",
"language": "python",
"name": "python3"
},
"language_info": {
"codemirror_mode": {
"name": "ipython",
"version": 3
},
"file_extension": ".py",
"mimetype": "text/x-python",
"name": "python",
"nbconvert_exporter": "python",
"pygments_lexer": "ipython3",
"version": "3.8.18"
},
"required_libs": []
},
"nbformat": 4,
"nbformat_minor": 5
}
常见的CNN模型/ResNet_mxy/ResNet.pdf 0 → 100644
This source diff could not be displayed because it is too large. You can view the blob instead.
常见的CNN模型/ResNet_mxy/code/ResNet.html 0 → 100644
This source diff could not be displayed because it is too large. You can view the blob instead.
常见的CNN模型/UNet_wyx/code/__init__.py 0 → 100644
from .unet_model import UNet
from .unet_model import UNet
常见的CNN模型/UNet_wyx/code/mini_unet.py 0 → 100644
import torch
import torch
import torch.nn as nn
import torch.nn.functional as F
# 深层比较容易出现过拟合现象，因此在深层开启dropout规则
def DropOrPass(do_dropout):
if do_dropout:
return nn.Dropout2d()
return (lambda x, **kwargs: x)
# 添加了对输入维度的检查，确保输入为4维（样本x通道x图像）
class ContBatchNorm2d(nn.modules.batchnorm._BatchNorm):
def _check_input_dim(self, input):
if input.dim() != 4:
raise ValueError('expected 4D input (got {}D input)'
.format(input.dim()))
def forward(self, input):
self._check_input_dim(input)
return F.batch_norm(
input, self.running_mean, self.running_var, self.weight, self.bias,
True, self.momentum, self.eps)
# 单个卷积、批归一、非线性一条龙
class ConvLayer(nn.Module):
def __init__(self, nchan):
super(ConvLayer, self).__init__()
self.conv = nn.Conv2d(nchan, nchan, kernel_size=5, padding=2)
self.bn = ContBatchNorm2d(nchan)
self.relu = nn.ELU(inplace=True)
def forward(self, x):
out = self.relu(self.bn(self.conv(x)))
return out
# 连续进行depth个一条龙
def _ConvSeq(nchan, depth):
return nn.Sequential(*[ConvLayer(nchan) for _ in range(depth)])
# in_tr
class InTrans(nn.Module):
def __init__(self, ochan):
super(InTrans, self).__init__()
self.ochan = ochan
# kernel_size=5, padding=2 确保输入输出等宽可以直接相加
self.conv = nn.Conv2d(1, ochan, kernel_size=5, padding=2)
self.bn = ContBatchNorm2d(ochan)
self.relu = nn.ELU(inplace=True)
def forward(self, x):
fx = self.bn(self.conv(x))
x_res = torch.cat([x for _ in range(self.ochan)], 1)
out = self.relu(torch.add(fx, x_res))
return out
# down_tr 输出将分别送往下层的down_tr与同层的up_tr
class DownTrans(nn.Module):
def __init__(self, ichan, conv_depth, dropout = False):
super(DownTrans, self).__init__()
ochan = 2 * ichan
# 下采样部分
self.conv_down = nn.Conv2d(ichan, ochan, kernel_size=2, stride=2)
self.bn = ContBatchNorm2d(ochan)
self.relu1 = nn.ELU(inplace=True)
# 卷积序列部分
self.drop = DropOrPass(dropout)
self.conv_seq = _ConvSeq(ochan, conv_depth)
# 残差部分
self.relu2 = nn.ELU(inplace=True)
def forward(self, x):
x_down = self.relu1(self.bn(self.conv_down(x)))
fx = self.conv_seq(self.drop(x_down))
out = self.relu2(torch.add(fx, x_down))
return out
# up_tr 接受来自下层的up_tr与同层的down_tr输入
class UpTrans(nn.Module):
def __init__(self, ichan, ochan, conv_depth, dropout = False):
super(UpTrans, self).__init__()
schan = ochan // 2 # 同层skip输入通道数
# 上采样部分
self.conv_up = nn.ConvTranspose2d(ichan, schan, kernel_size=2, stride=2)
self.bn = ContBatchNorm2d(schan)
self.relu1 = nn.ELU(inplace = True)
# 跳跃连接部分
self.drop1 = DropOrPass(dropout)
self.drop2 = nn.Dropout2d() # 始终默认Dropout
# 卷积序列部分
self.conv_seq = _ConvSeq(ochan, conv_depth)
# 残差部分
self.relu2 = nn.ELU(inplace = True)
def forward(self, x, x_skip):
x_up = self.relu1(self.bn(self.conv_up(self.drop1(x)))) # 上采样
x_cat = torch.cat((x_up, self.drop2(x_skip)), 1) # 跳跃连接
fx = self.conv_seq(x_cat) # 卷积序列
out = self.relu2(torch.add(fx, x_cat)) # 残差
return out
# out_tr
class OutTrans(nn.Module):
def __init__(self, ichan, nll):
super(OutTrans, self).__init__()
self.conv1 = nn.Conv2d(ichan, 2, kernel_size=5, padding=2)
self.bn = ContBatchNorm2d(2)
self.conv2 = nn.Conv2d(2, 2, kernel_size = 1)
self.relu = nn.ELU(inplace = True)
self.softmax = F.log_softmax if nll else F.softmax
def forward(self, x):
out = self.conv2(self.relu(self.bn(self.conv1(x))))
# 将通道转置到最后一维
out = out.permute(0, 2, 3, 4, 1).contiguous()
out = out.view(out.numel() // 2, 2)
out = self.softmax(out)
return out
# out_tr_same
class OutTransSame(nn.Module):
def __init__(self, ichan, nll):
super(OutTransSame, self).__init__()
self.conv1 = nn.Conv2d(ichan, 1, kernel_size=5, padding=2)
self.bn = ContBatchNorm2d(1)
self.conv2 = nn.Conv2d(1, 1, kernel_size = 1)
self.relu = nn.ELU(inplace = True)
def forward(self, x):
out = self.conv2(self.relu(self.bn(self.conv1(x))))
return out
class UNet(nn.Module):
def __init__(self, nll = False):
super(UNet, self).__init__()
self.in_tr = InTrans(16) # 1->16
self.down_tr1 = DownTrans(16, 1) # 16->32
self.down_tr2 = DownTrans(32, 2) # 32->64
self.down_tr3 = DownTrans(64, 3, dropout = True) # 64->128
#self.down_tr4 = DownTrans(128, 2, dropout = True) # 128->256
#self.up_tr43 = UpTrans(256, 256, 2, dropout = True) # 256->128(+128)->256
#self.up_tr32 = UpTrans(256, 128, 2, dropout = True) # 256->64(+64)->128
self.up_tr32 = UpTrans(128, 128, 2, dropout = True) # 128->64(+64)->128
self.up_tr21 = UpTrans(128, 64, 1, dropout = True) # 128->32(+32)->64
self.up_tr10 = UpTrans(64, 32, 1) # 64->16(+16)->32
self.out_tr = OutTransSame(32, nll) # 32->1
def forward(self, x):
fx0 = self.in_tr(x)
fx1 = self.down_tr1(fx0)
fx2 = self.down_tr2(fx1)
fx3 = self.down_tr3(fx2)
#fx4 = self.down_tr4(fx3)
#out = self.up_tr43(fx4, fx3)
#out = self.up_tr32(out, fx2)
out = self.up_tr32(fx3, fx2)
out = self.up_tr21(out, fx1)
out = self.up_tr10(out, fx0)
out = self.out_tr(out)
return out
if __name__ == "__main__":
net = UNet(1)
print(net)
\ No newline at end of file
常见的CNN模型/UNet_wyx/code/unet_model.py 0 → 100644
""" Full assembly of the parts to form the complete network """
""" Full assembly of the parts to form the complete network """
from unet_parts import *
class UNet(nn.Module):
def __init__(self, n_channels, n_classes):
super(UNet, self).__init__()
self.n_channels = n_channels
self.n_classes = n_classes
self.inc = DoubleConv(n_channels, 64)
self.down1 = Down(64, 128)
self.down2 = Down(128, 256)
self.down3 = Down(256, 512)
self.down4 = Down(512, 1024)
self.up1 = Up(1024, 512)
self.up2 = Up(512, 256)
self.up3 = Up(256, 128)
self.up4 = Up(128, 64)
self.outc = OutConv(64, n_classes)
def forward(self, x):
x1 = self.inc(x)
x2 = self.down1(x1)
x3 = self.down2(x2)
x4 = self.down3(x3)
x5 = self.down4(x4)
x = self.up1(x5, x4)
x = self.up2(x, x3)
x = self.up3(x, x2)
x = self.up4(x, x1)
final = self.outc(x)
return final
if __name__ == "__main__":
net = UNet(3,1)
print(net)
\ No newline at end of file
常见的CNN模型/UNet_wyx/code/unet_parts.py 0 → 100644
""" Parts of the U-Net model """
""" Parts of the U-Net model """
import torch
import torch.nn as nn
import torch.nn.functional as F
class DoubleConv(nn.Module):
"""(convolution => [BN] => ReLU) * 2"""
def __init__(self, in_channels, out_channels):
super().__init__()
self.double_conv = nn.Sequential(
nn.Conv2d(in_channels, out_channels, kernel_size=3, padding=1, bias=False),
nn.BatchNorm2d(out_channels),
nn.ReLU(inplace=True),
nn.Conv2d(out_channels, out_channels, kernel_size=3, padding=1, bias=False),
nn.BatchNorm2d(out_channels),
nn.ReLU(inplace=True)
)
def forward(self, x):
return self.double_conv(x)
class Down(nn.Module):
"""Downscaling with maxpool then double conv"""
def __init__(self, in_channels, out_channels):
super().__init__()
#下采样和双卷积
self.maxpool_conv = nn.Sequential(
nn.MaxPool2d(2),
DoubleConv(in_channels, out_channels)
)
def forward(self, x):
return self.maxpool_conv(x)
class Up(nn.Module):
"""Upscaling then double conv"""
def __init__(self, in_channels, out_channels):
super().__init__()
# 使用转置卷积进行上采样
self.up = nn.ConvTranspose2d(in_channels, in_channels // 2, kernel_size=2, stride=2)
# 双卷积操作
self.conv = DoubleConv(in_channels, out_channels)
def forward(self, x1, x2):
# 上采样
x1 = self.up(x1)
# 将 x2 和调整后的 x1 连接在一起
x = torch.cat([x2, x1], dim=1)
# 对连接后的张量进行双卷积操作
return self.conv(x)
class OutConv(nn.Module):
def __init__(self, in_channels, out_channels):
super(OutConv, self).__init__()
#最终输出分割结果
self.conv = nn.Conv2d(in_channels, out_channels, kernel_size=1)
def forward(self, x):
return self.conv(x)
机器学习基础mxy/K_Means可视化网址.txt 0 → 100644
https://www.naftaliharris.com/blog/visualizing-k-means-clustering/
https://www.naftaliharris.com/blog/visualizing-k-means-clustering/
\ No newline at end of file
机器学习基础mxy/PCA的数学原理.mhtml 0 → 100644
This source diff could not be displayed because it is too large. You can view the blob instead.
机器学习基础mxy/内容.txt 0 → 100644
引入
引入
过拟合 欠拟合
No Free Lunch
交叉验证
KNN（有监督）
决策树（有监督）
Kmeans（无监督）
PCA（无监督）
\ No newline at end of file
机器学习基础mxy/提纲.txt 0 → 100644
人工智能 机器学习 深度学习
人工智能 机器学习 深度学习
过拟合 欠拟合
交叉验证
KNN(虽然名字带有nn，但属于分类算法，K-Nearest-Neighbors,近朱者赤近墨者黑）
算法关键：
（1）样本的所有特征都要做可比较的量化。若是样本特征中存在非数值的类型，必须采取手段将其量化为数值。例如样本特征中包含颜色，可通过颜色转换为灰度值来实现距离计算。
（2）样本特征要做归一化处理。样本有多个参数，每一个参数都有自己的定义域和取值范围，他们对距离计算的影响不一样，如取值较大的影响力会盖过取值较小的参数。
（3）需要一个距离函数以计算两个样本之间的距离。欧氏距离和曼哈顿距离。
（4）确定K的值。太大容易欠拟合，太小容易过拟合，需交叉验证确定K值。
优点：易于理解，无需估计参数，无需训练。特别适合多分类问题。
缺点：当样本不平衡时，如一个类的样本容量很大，而其他类样本容量很小时，有可能导致当输入一个新样本时，该样本的K个邻居中大容量类的样本占多数，如下图所示。（可以采用权值的方法改进）
计算量较大，因为对每一个待分类的文本都要计算它到全体已知样本的距离，才能求得它的K个最近邻点。
算法步骤：（1）算距离；（2）找邻居；（3）做分类。
决策树（一种基本的分类与回归方法，主要讨论分类）
if-then集合
熵：熵是表示随机变量不确定性的度量（说白了就是物体内部的混乱程度，对比义乌和苹果专卖店）
信息增益大＝熵值下降多
算法步骤：（1）特征选择；（2）决策树的生成；（3）决策树的修剪。
算法本质：从训练集中归纳出一组分类规则，或者说是由训练数据集估计条件概率模型。
注意：
（1）决策树对特征选择的顺序十分严格，如果顺序不一样，最后得到的分类结果会有误差
决策树的训练与测试
训练阶段：从给定的训练集构造出来一棵树（从根节点开始选择特征，如何进行特征切分）
测试阶段：根据构造出来的树模型从上到下去走一遍
一旦构造好了决策树，那么分类或者预测任务就很简单，只要走一遍就可以了，难点在于如何构造出一颗决策树，包括特征选择、阈值设置等等问题。
决策树的剪枝
为什么要剪枝？决策树过拟合风险很大，理论上可以完全分得开数据
剪枝策略：预剪枝，后剪枝
预剪枝：一边建立决策树一边进行剪枝的操作（一般都用预剪枝）。限制深度，叶子节点个数、叶子结点样本数、信息增益量等。
后剪枝：当建立完决策树后进行剪枝操作。叶子节点越多，损失越大。
PCA
PCA（Principal Component Analysis） 是一种常见的数据分析方式，常用于高维数据的降维，可用于提取数据的主要特征分量。
多变量大样本无疑会为研究和应用提供了丰富的信息，但也在一定程度上增加了数据采集的工作量，更重要的是在多数情况下，许多变量之间可能存在相关性，从而增加了问题分析的复杂性，同时对分析带来不便。如果分别对每个指标进行分析，分析往往是孤立的，而不是综合的。盲目减少指标会损失很多信息，容易产生错误的结论。
所以需要找到一个合理的方法，在减少需要分析的指标同时，尽量减少原指标包含信息的损失，以达到对所收集数据进行全面分析的目的。
去中心化不会影响样本的分布性质，但会简化PCA算法的推导过程。
PCA可以帮助提取数据的关键特征并简化数据集，从而降低数据的复杂度，加快机器学习算法的训练速度，并降低存储空间要求。
数据降维是怎么回事儿？假设三维空间中有一系列点，这些点分布在一个过原点的斜面上，如果你用自然坐标系x,y,z这三个轴来表示这组数据的话，需要使用三个维度，而事实上，这些点的分布仅仅是在一个二维的平面上，那么，问题出在哪里？如果你再仔细想想，能不能把x,y,z坐标系旋转一下，使数据所在平面与x,y平面重合？这就对了！如果把旋转后的坐标系记为x',y',z'，那么这组数据的表示只用x'和y'两个维度表示即可！当然了，如果想恢复原来的表示方式，那就得把这两个坐标之间的变换矩阵存下来。这样就能把数据维度降下来了！
PCA的思想是将n维特征映射到k维上（k<n），这k维是全新的正交特征。这k维特征称为主成分，是重新构造出来的k维特征，而不是简单地从n维特征中去除其余n-k维特征。
K-Menas
对初始种子很敏感
\ No newline at end of file
Markdown 格式
0% 或
您添加了 0 人 到此讨论。请谨慎行事。
请先完成此评论的编辑！
请 注册 或者 后发表评论