求函数最小值-torch版

目标：torch实现下面链接中的梯度下降法

先计算  的导函数  ，然后计算导函数 在处的梯度 (导数)

让 沿着 梯度的负方向移动，

自变量 的更新过程如下

torch代码实现如下

import torch
x = torch.tensor([7.5],requires_grad=True)
# print(x.grad)
optimizer = torch.optim.SGD([x], lr=1)
print('x_0 = {}'.format(x))
for i in range(10):
    y = x * x
    optimizer.zero_grad()
    y.backward()
    optimizer.step()
    print('x_{} = {}'.format(i+1,x))

运行效果如下：

x_0 = tensor([7.5000], requires_grad=True)
x_1 = tensor([-7.5000], requires_grad=True)
x_2 = tensor([7.5000], requires_grad=True)
x_3 = tensor([-7.5000], requires_grad=True)
x_4 = tensor([7.5000], requires_grad=True)
x_5 = tensor([-7.5000], requires_grad=True)
x_6 = tensor([7.5000], requires_grad=True)
x_7 = tensor([-7.5000], requires_grad=True)
x_8 = tensor([7.5000], requires_grad=True)
x_9 = tensor([-7.5000], requires_grad=True)
x_10 = tensor([7.5000], requires_grad=True)

给梯度加系数

我们可以给 梯度 加个系数，如下

torch代码实现如下

import torch
x = torch.tensor([7.5],requires_grad=True)
# print(x.grad)
optimizer = torch.optim.SGD([x], lr=0.01)
print('x_0 = {}'.format(x))
for i in range(10):
    y = x * x
    optimizer.zero_grad()
    y.backward()
    optimizer.step()
    print('x_{} = {}'.format(i+1,x))

运行效果如下：

x_0 = tensor([7.5000], requires_grad=True)
x_1 = tensor([7.3500], requires_grad=True)
x_2 = tensor([7.2030], requires_grad=True)
x_3 = tensor([7.0589], requires_grad=True)
x_4 = tensor([6.9178], requires_grad=True)
x_5 = tensor([6.7794], requires_grad=True)
x_6 = tensor([6.6438], requires_grad=True)
x_7 = tensor([6.5109], requires_grad=True)
x_8 = tensor([6.3807], requires_grad=True)
x_9 = tensor([6.2531], requires_grad=True)
x_10 = tensor([6.1280], requires_grad=True)

调迭代次数

发现 变化的很慢，我们可以增加迭代次数，如下

import torch
x = torch.tensor([7.5],requires_grad=True)
# print(x.grad)
optimizer = torch.optim.SGD([x], lr=0.01)
print('x_0 = {}'.format(x))
for i in range(200):
    y = x * x
    optimizer.zero_grad()
    y.backward()
    optimizer.step()
    print('x_{} = {}'.format(i+1,x))

运行结果如下：

x_0 = tensor([7.5000], requires_grad=True)
x_1 = tensor([7.3500], requires_grad=True)
x_2 = tensor([7.2030], requires_grad=True)
...
x_199 = tensor([0.1346], requires_grad=True)
x_200 = tensor([0.1319], requires_grad=True)

调梯度系数

我们把 0.01 换成 0.1 试试

import torch
x = torch.tensor([7.5],requires_grad=True)
# print(x.grad)
optimizer = torch.optim.SGD([x], lr=0.1)
print('x_0 = {}'.format(x))
for i in range(10):
    y = x * x
    optimizer.zero_grad()
    y.backward()
    optimizer.step()
    print('x_{} = {}'.format(i+1,x))

运行结果如下：

x_0 = tensor([7.5000], requires_grad=True)
x_1 = tensor([6.], requires_grad=True)
x_2 = tensor([4.8000], requires_grad=True)
x_3 = tensor([3.8400], requires_grad=True)
x_4 = tensor([3.0720], requires_grad=True)
x_5 = tensor([2.4576], requires_grad=True)
x_6 = tensor([1.9661], requires_grad=True)
x_7 = tensor([1.5729], requires_grad=True)
x_8 = tensor([1.2583], requires_grad=True)
x_9 = tensor([1.0066], requires_grad=True)
x_10 = tensor([0.8053], requires_grad=True)