-
Notifications
You must be signed in to change notification settings - Fork 53
Open
Description
I think the function of initialize_weight can be:
def initialize_weights(*models):
for model in models:
for m in model.modules():
if isinstance(m, nn.Conv2d):
nn.init.kaiming_normal_(m.weight.data, nonlinearity='relu')
elif isinstance(m, nn.BatchNorm2d):
m.weight.data.fill_(1.)
m.bias.data.fill_(1e-4)
elif isinstance(m, nn.Linear):
m.weight.data.normal_(0.0, 0.0001)
m.bias.data.zero_()
While using ReLU, Kaiming initializing is better.
Reactions are currently unavailable
Metadata
Metadata
Assignees
Labels
No labels