Nameexact_genome_1542189904_172_5894_2
Workunit4088646
Created15 Nov 2018, 11:35:50 UTC
Sent15 Nov 2018, 11:36:22 UTC
Report deadline9 Dec 2018, 4:15:07 UTC
Received16 Nov 2018, 7:56:12 UTC
Server stateOver
OutcomeSuccess
Client stateDone
Exit status0 (0x0)
Computer ID62723
Run time8 hours 53 min 52 sec
CPU time8 hours 28 min 42 sec
Validate stateValid
Credit5,923.40
Device peak FLOPS4.38 GFLOPS
Application versionEXACT Batch Norm With Scaled FMP CNN Trainer v0.34
Peak working set size564.70 MB
Peak swap size584.46 MB
Peak disk usage11.92 MB

Stderr output

<core_client_version>7.6.33</core_client_version>
<![CDATA[
<stderr_txt>
arguments:
	'projects/csgrid.org_csg/exact_client_0.34_windows_x86_64.exe'
	'--training_file'
	'training_samples.bin'
	'--validation_file'
	'validation_samples.bin'
	'--testing_file'
	'testing_samples.bin'
	'--genome_file'
	'input_genome.txt'
	'--output_file'
	'output_genome.txt'
	'--checkpoint_file'
	'checkpoint.txt'
converting arguments to vector
boincified training filename: '../../projects/csgrid.org_csg/cifar_10_training.bin'
boincified validation filename: '../../projects/csgrid.org_csg/cifar_10_testing.bin'
boincified testing filename: '../../projects/csgrid.org_csg/cifar_10_testing.bin'
boincified genome filename: '../../projects/csgrid.org_csg/exact_genome_1542189904_172_5894.txt'
boincified output filename: '../../projects/csgrid.org_csg/exact_genome_1542189904_172_5894_2_r1968819509_0'
boincified checkpoint filename: 'checkpoint.txt'
loading genome
starting from input file: '../../projects/csgrid.org_csg/exact_genome_1542189904_172_5894.txt'
read CNN_Genome file with version string: 'v0.33'
read exact_id: -1
read genome_id: -1
read initial_mu: 0.493958
read mu: 0.493958
read mu_delta: 0.937178
read initial_learning_rate: 1e-05
read learning_rate: 1e-05
read learning_rate_delta: 0.928176
read initial_weight_decay: 0.000233202
read weight_decay: 0.000233202
read weight_decay_delta: 0.941751
read batch_size: 103
read epsilon: 1e-07
read alpha: 0.0952689
read input_dropout_probability: 0.00136406
read hidden_dropout_probability: 0.0828725
read velocity_reset: 1815
read epoch: 0
read max_epochs: 25
read reset_weights: 0
read padding: 2
read best_epoch: 0
read number_validation_images: 10000
read best_validation_predictions: 0
read best_validation_error: 1e+07
read number_training_images: 50000
read training_predictions: 0
read training_error: 1e+07
read number_test_images: 10000
read test_predictions: 0
read test_error: 1e+07
read generation_id: 5894
read normal distribution: '1 1.3258730173111 -1.91948688030243'
generator_str: '1817215583'
read generator: 1817215583
read generated_by_map:
1 crossover 1
reading 93 nodes.
reading 387 edges.
number input nodes: 3
number softmax nodes: 10
order_size: 0
parsed input file
loaded genome
loading images
number_classes: 10
channels: 3
width: 32
height: 32
reading image set with 5000 images.
reading image set with 5000 images.
reading image set with 5000 images.
reading image set with 5000 images.
reading image set with 5000 images.
reading image set with 5000 images.
reading image set with 5000 images.
reading image set with 5000 images.
reading image set with 5000 images.
reading image set with 5000 images.
image_size: 3x32x32 = 3072
read 50000 images.
    class    0: 5000
    class    1: 5000
    class    2: 5000
    class    3: 5000
    class    4: 5000
    class    5: 5000
    class    6: 5000
    class    7: 5000
    class    8: 5000
    class    9: 5000
calculating averages and standard deviations for images
average pixel value for channel 0: 0.491399973630905
average pixel value for channel 1: 0.482159107923508
average pixel value for channel 2: 0.446530401706696
pixel variance for channel 0: 0.0610253289341927
pixel standard deviation for channel 0: 0.247033059597015
pixel variance for channel 1: 0.0592849254608154
pixel standard deviation for channel 1: 0.24348495900631
pixel variance for channel 2: 0.0684280395507813
pixel standard deviation for channel 2: 0.261587530374527
number_classes: 10
channels: 3
width: 32
height: 32
reading image set with 1000 images.
reading image set with 1000 images.
reading image set with 1000 images.
reading image set with 1000 images.
reading image set with 1000 images.
reading image set with 1000 images.
reading image set with 1000 images.
reading image set with 1000 images.
reading image set with 1000 images.
reading image set with 1000 images.
image_size: 3x32x32 = 3072
read 10000 images.
    class    0: 1000
    class    1: 1000
    class    2: 1000
    class    3: 1000
    class    4: 1000
    class    5: 1000
    class    6: 1000
    class    7: 1000
    class    8: 1000
    class    9: 1000
number_classes: 10
channels: 3
width: 32
height: 32
reading image set with 1000 images.
reading image set with 1000 images.
reading image set with 1000 images.
reading image set with 1000 images.
reading image set with 1000 images.
reading image set with 1000 images.
reading image set with 1000 images.
reading image set with 1000 images.
reading image set with 1000 images.
reading image set with 1000 images.
image_size: 3x32x32 = 3072
read 10000 images.
    class    0: 1000
    class    1: 1000
    class    2: 1000
    class    3: 1000
    class    4: 1000
    class    5: 1000
    class    6: 1000
    class    7: 1000
    class    8: 1000
    class    9: 1000
loaded images
starting backpropagation!
generator min: 1, generator max: 2147483646
pre shuffle 1: 429875847
post shuffle 1: 1735691757
epoch time: 1077.13818359375s, input_fired_time: 16.8014450073242, output_fired_time: 8.39621734619141, propagate_forward_time: 411.096221923828, propagate_backward_time: 631.163818359375, weight_update_time: 0.303798049688339, other_time: 9.6805419921875
epoch time: 85.6109237670898s, input_fired_time: 1.33704793453217, output_fired_time: 0, propagate_forward_time: 82.6034622192383, propagate_backward_time: 0, weight_update_time: 0, other_time: 1.67041015625
validation[          , genome  5894] predictions:    6339/  10000 (63.39%), best:    6339/10000 (63.39%), error:     10747.15625, best error:     10747.15625 on epoch:     0, epoch:    0/25, mu: 0.4939583540, learning_rate: 0.0000100000, weight_decay: 0.0002332021

epoch time: 1105.3288574219s, input_fired_time: 17.4211502075, output_fired_time: 8.7359209061, propagate_forward_time: 420.0029296875, propagate_backward_time: 648.9793701172, weight_update_time: 0.3187949657, other_time: 10.1894531250
epoch time: 87.8562622070s, input_fired_time: 1.4154648781, output_fired_time: 0.0000000000, propagate_forward_time: 84.6383514404, propagate_backward_time: 0.0000000000, weight_update_time: 0.0000000000, other_time: 1.8024444580
validation[          , genome  5894] predictions:    6381/  10000 (63.81%), best:    6339/10000 (63.39%), error:     10759.76660, best error:     10747.15625 on epoch:     0, epoch:    1/25, mu: 0.5251207948, learning_rate: 0.0000092818, weight_decay: 0.0002196183

epoch time: 1096.2341308594s, input_fired_time: 17.3249816895, output_fired_time: 8.6637887955, propagate_forward_time: 414.4674682617, propagate_backward_time: 645.6188354492, weight_update_time: 0.3121324778, other_time: 10.1590576172
epoch time: 86.7561111450s, input_fired_time: 1.3812936544, output_fired_time: 0.0000000000, propagate_forward_time: 83.6847991943, propagate_backward_time: 0.0000000000, weight_update_time: 0.0000000000, other_time: 1.6900177002
validation[          , genome  5894] predictions:    6366/  10000 (63.66%), best:    6339/10000 (63.39%), error:     10894.97656, best error:     10747.15625 on epoch:     0, epoch:    2/25, mu: 0.5543255806, learning_rate: 0.0000086151, weight_decay: 0.0002068257

epoch time: 1080.2135009766s, input_fired_time: 16.8725471497, output_fired_time: 8.4197940826, propagate_forward_time: 412.1465454102, propagate_backward_time: 633.0486450195, weight_update_time: 0.3051073253, other_time: 9.7259521484
epoch time: 85.9483642578s, input_fired_time: 1.3503595591, output_fired_time: 0.0000000000, propagate_forward_time: 82.8814468384, propagate_backward_time: 0.0000000000, weight_update_time: 0.0000000000, other_time: 1.7165603638
validation[          , genome  5894] predictions:    6364/  10000 (63.64%), best:    6339/10000 (63.39%), error:     11045.94238, best error:     10747.15625 on epoch:     0, epoch:    3/25, mu: 0.5816956162, learning_rate: 0.0000079963, weight_decay: 0.0001947783

epoch time: 1079.6088867188s, input_fired_time: 16.8473758698, output_fired_time: 8.4315462112, propagate_forward_time: 411.7477722168, propagate_backward_time: 632.8155517578, weight_update_time: 0.3058041036, other_time: 9.7666015625
epoch time: 86.2739944458s, input_fired_time: 1.3596817255, output_fired_time: 0.0000000000, propagate_forward_time: 83.2071609497, propagate_backward_time: 0.0000000000, weight_update_time: 0.0000000000, other_time: 1.7071533203
validation[          , genome  5894] predictions:    6316/  10000 (63.16%), best:    6339/10000 (63.39%), error:     11357.67188, best error:     10747.15625 on epoch:     0, epoch:    4/25, mu: 0.6073462367, learning_rate: 0.0000074220, weight_decay: 0.0001834327

epoch time: 1081.8200683594s, input_fired_time: 16.8866882324, output_fired_time: 8.4518089294, propagate_forward_time: 412.9491271973, propagate_backward_time: 633.7191162109, weight_update_time: 0.3057778180, other_time: 9.8133544922
epoch time: 85.9966659546s, input_fired_time: 1.3527137041, output_fired_time: 0.0000000000, propagate_forward_time: 82.9220733643, propagate_backward_time: 0.0000000000, weight_update_time: 0.0000000000, other_time: 1.7218780518
validation[          , genome  5894] predictions:    6363/  10000 (63.63%), best:    6339/10000 (63.39%), error:     11436.43457, best error:     10747.15625 on epoch:     0, epoch:    5/25, mu: 0.6313854456, learning_rate: 0.0000068889, weight_decay: 0.0001727479

epoch time: 1089.4396972656s, input_fired_time: 17.2290477753, output_fired_time: 8.4902410507, propagate_forward_time: 413.1282348633, propagate_backward_time: 640.5356445313, weight_update_time: 0.3094149232, other_time: 10.0565795898
epoch time: 88.4766387939s, input_fired_time: 1.3641811609, output_fired_time: 0.0000000000, propagate_forward_time: 85.2864303589, propagate_backward_time: 0.0000000000, weight_update_time: 0.0000000000, other_time: 1.8260269165
validation[          , genome  5894] predictions:    6313/  10000 (63.13%), best:    6339/10000 (63.39%), error:     11697.83008, best error:     10747.15625 on epoch:     0, epoch:    6/25, mu: 0.6539144516, learning_rate: 0.0000063942, weight_decay: 0.0001626855

epoch time: 1081.1794433594s, input_fired_time: 17.0188732147, output_fired_time: 8.5642976761, propagate_forward_time: 411.0904235840, propagate_backward_time: 634.6577758789, weight_update_time: 0.3049608767, other_time: 9.8479614258
epoch time: 87.9922790527s, input_fired_time: 1.3382030725, output_fired_time: 0.0000000000, propagate_forward_time: 84.9491043091, propagate_backward_time: 0.0000000000, weight_update_time: 0.0000000000, other_time: 1.7049713135
validation[          , genome  5894] predictions:    6283/  10000 (62.83%), best:    6339/10000 (63.39%), error:     11849.45410, best error:     10747.15625 on epoch:     0, epoch:    7/25, mu: 0.6750280857, learning_rate: 0.0000059349, weight_decay: 0.0001532092

epoch time: 1088.2778320313s, input_fired_time: 17.1248416901, output_fired_time: 8.4755992889, propagate_forward_time: 413.1555786133, propagate_backward_time: 639.6282348633, weight_update_time: 0.3076821864, other_time: 9.8935546875
epoch time: 86.6017761230s, input_fired_time: 1.3564699888, output_fired_time: 0.0000000000, propagate_forward_time: 83.5158157349, propagate_backward_time: 0.0000000000, weight_update_time: 0.0000000000, other_time: 1.7294921875
validation[          , genome  5894] predictions:    6276/  10000 (62.76%), best:    6339/10000 (63.39%), error:     11984.79004, best error:     10747.15625 on epoch:     0, epoch:    8/25, mu: 0.6948153377, learning_rate: 0.0000055086, weight_decay: 0.0001442849

epoch time: 1077.0152587891s, input_fired_time: 16.7565078735, output_fired_time: 8.3813543320, propagate_forward_time: 410.8928222656, propagate_backward_time: 631.1934814453, weight_update_time: 0.3036642969, other_time: 9.7911376953
epoch time: 86.1665115356s, input_fired_time: 1.3432736397, output_fired_time: 0.0000000000, propagate_forward_time: 83.1097106934, propagate_backward_time: 0.0000000000, weight_update_time: 0.0000000000, other_time: 1.7135238647
validation[          , genome  5894] predictions:    6234/  10000 (62.34%), best:    6339/10000 (63.39%), error:     12155.48242, best error:     10747.15625 on epoch:     0, epoch:    9/25, mu: 0.7133595347, learning_rate: 0.0000051130, weight_decay: 0.0001358804

epoch time: 1081.9836425781s, input_fired_time: 16.8606166840, output_fired_time: 8.4422779083, propagate_forward_time: 412.8798217773, propagate_backward_time: 634.0294189453, weight_update_time: 0.3028718233, other_time: 9.7715454102
epoch time: 85.9440078735s, input_fired_time: 1.3406498432, output_fired_time: 0.0000000000, propagate_forward_time: 82.8878860474, propagate_backward_time: 0.0000000000, weight_update_time: 0.0000000000, other_time: 1.7154693604
validation[          , genome  5894] predictions:    6190/  10000 (61.90%), best:    6339/10000 (63.39%), error:     12320.54785, best error:     10747.15625 on epoch:     0, epoch:   10/25, mu: 0.7307386994, learning_rate: 0.0000047458, weight_decay: 0.0001279655

epoch time: 1078.2569580078s, input_fired_time: 16.7794780731, output_fired_time: 8.4191675186, propagate_forward_time: 411.2762756348, propagate_backward_time: 632.0003051758, weight_update_time: 0.3030690849, other_time: 9.7817993164
epoch time: 86.0619888306s, input_fired_time: 1.3507012129, output_fired_time: 0.0000000000, propagate_forward_time: 82.9904937744, propagate_backward_time: 0.0000000000, weight_update_time: 0.0000000000, other_time: 1.7207946777
validation[          , genome  5894] predictions:    6185/  10000 (61.85%), best:    6339/10000 (63.39%), error:     12403.23145, best error:     10747.15625 on epoch:     0, epoch:   11/25, mu: 0.7470260859, learning_rate: 0.0000044049, weight_decay: 0.0001205116

epoch time: 1235.8085937500s, input_fired_time: 19.2190589905, output_fired_time: 9.9238367081, propagate_forward_time: 469.1666259766, propagate_backward_time: 726.1272583008, weight_update_time: 0.3777109087, other_time: 11.3717651367
epoch time: 85.6471557617s, input_fired_time: 1.3367328644, output_fired_time: 0.0000000000, propagate_forward_time: 82.6045761108, propagate_backward_time: 0.0000000000, weight_update_time: 0.0000000000, other_time: 1.7058486938
validation[          , genome  5894] predictions:    6176/  10000 (61.76%), best:    6339/10000 (63.39%), error:     12576.63379, best error:     10747.15625 on epoch:     0, epoch:   12/25, mu: 0.7622902393, learning_rate: 0.0000040885, weight_decay: 0.0001134919

epoch time: 1106.8548583984s, input_fired_time: 17.2157764435, output_fired_time: 8.7204790115, propagate_forward_time: 422.4881591797, propagate_backward_time: 648.5429687500, weight_update_time: 0.3020150959, other_time: 9.8874511719
epoch time: 102.4476165771s, input_fired_time: 1.5789458752, output_fired_time: 0.0000000000, propagate_forward_time: 98.6809616089, propagate_backward_time: 0.0000000000, weight_update_time: 0.0000000000, other_time: 2.1877059937
validation[          , genome  5894] predictions:    6122/  10000 (61.22%), best:    6339/10000 (63.39%), error:     12660.24805, best error:     10747.15625 on epoch:     0, epoch:   13/25, mu: 0.7765954733, learning_rate: 0.0000037949, weight_decay: 0.0001068811

epoch time: 1087.2941894531s, input_fired_time: 16.9955024719, output_fired_time: 8.5074844360, propagate_forward_time: 414.6318969727, propagate_backward_time: 637.3494873047, weight_update_time: 0.3045511544, other_time: 9.8098754883
epoch time: 86.0657577515s, input_fired_time: 1.3427643776, output_fired_time: 0.0000000000, propagate_forward_time: 83.0079040527, propagate_backward_time: 0.0000000000, weight_update_time: 0.0000000000, other_time: 1.7150878906
validation[          , genome  5894] predictions:    6099/  10000 (60.99%), best:    6339/10000 (63.39%), error:     12749.28613, best error:     10747.15625 on epoch:     0, epoch:   14/25, mu: 0.7900020480, learning_rate: 0.0000035223, weight_decay: 0.0001006554

epoch time: 1082.6761474609s, input_fired_time: 16.8275184631, output_fired_time: 8.4491891861, propagate_forward_time: 412.8505554199, propagate_backward_time: 634.6612548828, weight_update_time: 0.3057808876, other_time: 9.8875732422
epoch time: 86.1633605957s, input_fired_time: 1.3489369154, output_fired_time: 0.0000000000, propagate_forward_time: 83.0874099731, propagate_backward_time: 0.0000000000, weight_update_time: 0.0000000000, other_time: 1.7270126343
validation[          , genome  5894] predictions:    6059/  10000 (60.59%), best:    6339/10000 (63.39%), error:     12820.90918, best error:     10747.15625 on epoch:     0, epoch:   15/25, mu: 0.8025663495, learning_rate: 0.0000032693, weight_decay: 0.0000947923

epoch time: 1082.6878662109s, input_fired_time: 16.8340797424, output_fired_time: 8.4495344162, propagate_forward_time: 413.0924987793, propagate_backward_time: 634.4771728516, weight_update_time: 0.3049862087, other_time: 9.8344726563
epoch time: 86.2504043579s, input_fired_time: 1.3437558413, output_fired_time: 0.0000000000, propagate_forward_time: 83.2077789307, propagate_backward_time: 0.0000000000, weight_update_time: 0.0000000000, other_time: 1.6988677979
validation[          , genome  5894] predictions:    6075/  10000 (60.75%), best:    6339/10000 (63.39%), error:     12878.66797, best error:     10747.15625 on epoch:     0, epoch:   16/25, mu: 0.8143413663, learning_rate: 0.0000030345, weight_decay: 0.0000892707

arguments:
	'projects/csgrid.org_csg/exact_client_0.34_windows_x86_64.exe'
	'--training_file'
	'training_samples.bin'
	'--validation_file'
	'validation_samples.bin'
	'--testing_file'
	'testing_samples.bin'
	'--genome_file'
	'input_genome.txt'
	'--output_file'
	'output_genome.txt'
	'--checkpoint_file'
	'checkpoint.txt'
converting arguments to vector
boincified training filename: '../../projects/csgrid.org_csg/cifar_10_training.bin'
boincified validation filename: '../../projects/csgrid.org_csg/cifar_10_testing.bin'
boincified testing filename: '../../projects/csgrid.org_csg/cifar_10_testing.bin'
boincified genome filename: '../../projects/csgrid.org_csg/exact_genome_1542189904_172_5894.txt'
boincified output filename: '../../projects/csgrid.org_csg/exact_genome_1542189904_172_5894_2_r1968819509_0'
boincified checkpoint filename: 'checkpoint.txt'
loading genome
starting from checkpoint file: 'checkpoint.txt'
read CNN_Genome file with version string: 'v0.33'
read exact_id: -1
read genome_id: -1
read initial_mu: 0.493958
read mu: 0.825377
read mu_delta: 0.937178
read initial_learning_rate: 1e-05
read learning_rate: 2.81656e-06
read learning_rate_delta: 0.928176
read initial_weight_decay: 0.000233202
read weight_decay: 8.40708e-05
read weight_decay_delta: 0.941751
read batch_size: 103
read epsilon: 1e-07
read alpha: 0.0952689
read input_dropout_probability: 0.00136406
read hidden_dropout_probability: 0.0828725
read velocity_reset: 1815
read epoch: 17
read max_epochs: 25
read reset_weights: 0
read padding: 2
read best_epoch: 0
read number_validation_images: 10000
read best_validation_predictions: 6339
read best_validation_error: 10747.2
read number_training_images: 50000
read training_predictions: 0
read training_error: 1e+07
read number_test_images: 10000
read test_predictions: 0
read test_error: 1e+07
read generation_id: 5894
read normal distribution: '1 1.3258730173111 -1.91948688030243'
generator_str: '507385603'
read generator: 507385603
read generated_by_map:
1 crossover 1
reading 93 nodes.
reading 387 edges.
number input nodes: 3
number softmax nodes: 10
order_size: 50000
parsed input file
loaded genome
loading images
number_classes: 10
channels: 3
width: 32
height: 32
reading image set with 5000 images.
reading image set with 5000 images.
reading image set with 5000 images.
reading image set with 5000 images.
reading image set with 5000 images.
reading image set with 5000 images.
reading image set with 5000 images.
reading image set with 5000 images.
reading image set with 5000 images.
reading image set with 5000 images.
image_size: 3x32x32 = 3072
read 50000 images.
    class    0: 5000
    class    1: 5000
    class    2: 5000
    class    3: 5000
    class    4: 5000
    class    5: 5000
    class    6: 5000
    class    7: 5000
    class    8: 5000
    class    9: 5000
calculating averages and standard deviations for images
average pixel value for channel 0: 0.491399973630905
average pixel value for channel 1: 0.482159107923508
average pixel value for channel 2: 0.446530401706696
pixel variance for channel 0: 0.0610253289341927
pixel standard deviation for channel 0: 0.247033059597015
pixel variance for channel 1: 0.0592849254608154
pixel standard deviation for channel 1: 0.24348495900631
pixel variance for channel 2: 0.0684280395507813
pixel standard deviation for channel 2: 0.261587530374527
number_classes: 10
channels: 3
width: 32
height: 32
reading image set with 1000 images.
reading image set with 1000 images.
reading image set with 1000 images.
reading image set with 1000 images.
reading image set with 1000 images.
reading image set with 1000 images.
reading image set with 1000 images.
reading image set with 1000 images.
reading image set with 1000 images.
reading image set with 1000 images.
image_size: 3x32x32 = 3072
read 10000 images.
    class    0: 1000
    class    1: 1000
    class    2: 1000
    class    3: 1000
    class    4: 1000
    class    5: 1000
    class    6: 1000
    class    7: 1000
    class    8: 1000
    class    9: 1000
number_classes: 10
channels: 3
width: 32
height: 32
reading image set with 1000 images.
reading image set with 1000 images.
reading image set with 1000 images.
reading image set with 1000 images.
reading image set with 1000 images.
reading image set with 1000 images.
reading image set with 1000 images.
reading image set with 1000 images.
reading image set with 1000 images.
reading image set with 1000 images.
image_size: 3x32x32 = 3072
read 10000 images.
    class    0: 1000
    class    1: 1000
    class    2: 1000
    class    3: 1000
    class    4: 1000
    class    5: 1000
    class    6: 1000
    class    7: 1000
    class    8: 1000
    class    9: 1000
loaded images
starting backpropagation!
epoch time: 1275.57678222656s, input_fired_time: 19.9645347595215, output_fired_time: 9.896484375, propagate_forward_time: 485.530029296875, propagate_backward_time: 748.672912597656, weight_update_time: 0.328529894351959, other_time: 11.5128784179688
epoch time: 87.2472534179688s, input_fired_time: 1.39878177642822, output_fired_time: 0, propagate_forward_time: 84.1382369995117, propagate_backward_time: 0, weight_update_time: 0, other_time: 1.71023559570313
validation[          , genome  5894] predictions:    6055/  10000 (60.55%), best:    6339/10000 (63.39%), error:     12885.22754, best error:     10747.15625 on epoch:     0, epoch:   17/25, mu: 0.8253766298, learning_rate: 0.0000028166, weight_decay: 0.0000840708

epoch time: 1080.8186035156s, input_fired_time: 16.8796405792, output_fired_time: 8.4450359344, propagate_forward_time: 411.6022338867, propagate_backward_time: 634.1117553711, weight_update_time: 0.3080905676, other_time: 9.7799072266
epoch time: 85.8701782227s, input_fired_time: 1.3752125502, output_fired_time: 0.0000000000, propagate_forward_time: 82.7863540649, propagate_backward_time: 0.0000000000, weight_update_time: 0.0000000000, other_time: 1.7086105347
validation[          , genome  5894] predictions:    6015/  10000 (60.15%), best:    6339/10000 (63.39%), error:     13013.66504, best error:     10747.15625 on epoch:     0, epoch:   18/25, mu: 0.8357186317, learning_rate: 0.0000026143, weight_decay: 0.0000791737

epoch time: 1074.9331054688s, input_fired_time: 16.8820457458, output_fired_time: 8.4362173080, propagate_forward_time: 409.8114013672, propagate_backward_time: 630.0856933594, weight_update_time: 0.3060871661, other_time: 9.7177734375
epoch time: 85.6561279297s, input_fired_time: 1.3610521555, output_fired_time: 0.0000000000, propagate_forward_time: 82.5922470093, propagate_backward_time: 0.0000000000, weight_update_time: 0.0000000000, other_time: 1.7028274536
validation[          , genome  5894] predictions:    5971/  10000 (59.71%), best:    6339/10000 (63.39%), error:     13058.09766, best error:     10747.15625 on epoch:     0, epoch:   19/25, mu: 0.8454109430, learning_rate: 0.0000024265, weight_decay: 0.0000745619

epoch time: 1129.3992919922s, input_fired_time: 17.7944087982, output_fired_time: 8.9436044693, propagate_forward_time: 429.5204162598, propagate_backward_time: 662.6449584961, weight_update_time: 0.3085085154, other_time: 10.4959106445
epoch time: 111.0218276978s, input_fired_time: 1.8738420010, output_fired_time: 0.0000000000, propagate_forward_time: 107.0167999268, propagate_backward_time: 0.0000000000, weight_update_time: 0.0000000000, other_time: 2.1311874390
validation[          , genome  5894] predictions:    5996/  10000 (59.96%), best:    6339/10000 (63.39%), error:     13049.52734, best error:     10747.15625 on epoch:     0, epoch:   20/25, mu: 0.8544943333, learning_rate: 0.0000022522, weight_decay: 0.0000702188

epoch time: 1191.3829345703s, input_fired_time: 18.6542968750, output_fired_time: 9.5846633911, propagate_forward_time: 451.2351379395, propagate_backward_time: 701.1275024414, weight_update_time: 0.3808272779, other_time: 10.7813110352
epoch time: 113.2391815186s, input_fired_time: 1.9486110210, output_fired_time: 0.0000000000, propagate_forward_time: 108.8926544189, propagate_backward_time: 0.0000000000, weight_update_time: 0.0000000000, other_time: 2.3979187012
validation[          , genome  5894] predictions:    5990/  10000 (59.90%), best:    6339/10000 (63.39%), error:     13064.76172, best error:     10747.15625 on epoch:     0, epoch:   21/25, mu: 0.8630071282, learning_rate: 0.0000020905, weight_decay: 0.0000661286

epoch time: 1348.5388183594s, input_fired_time: 21.9680690765, output_fired_time: 11.1496515274, propagate_forward_time: 509.1437377930, propagate_backward_time: 793.4754028320, weight_update_time: 0.3930951953, other_time: 12.8020019531
epoch time: 109.4452896118s, input_fired_time: 1.6537407637, output_fired_time: 0.0000000000, propagate_forward_time: 105.5962677002, propagate_backward_time: 0.0000000000, weight_update_time: 0.0000000000, other_time: 2.1952819824
validation[          , genome  5894] predictions:    5975/  10000 (59.75%), best:    6339/10000 (63.39%), error:     13058.89453, best error:     10747.15625 on epoch:     0, epoch:   22/25, mu: 0.8709850907, learning_rate: 0.0000019403, weight_decay: 0.0000622767

epoch time: 1116.3289794922s, input_fired_time: 17.5529460907, output_fired_time: 8.7776184082, propagate_forward_time: 425.0254821777, propagate_backward_time: 654.7706298828, weight_update_time: 0.3201040328, other_time: 10.2022705078
epoch time: 88.6091995239s, input_fired_time: 1.4074703455, output_fired_time: 0.0000000000, propagate_forward_time: 85.4156951904, propagate_backward_time: 0.0000000000, weight_update_time: 0.0000000000, other_time: 1.7860336304
validation[          , genome  5894] predictions:    5910/  10000 (59.10%), best:    6339/10000 (63.39%), error:     13222.63379, best error:     10747.15625 on epoch:     0, epoch:   23/25, mu: 0.8784618974, learning_rate: 0.0000018010, weight_decay: 0.0000586491

epoch time: 1115.4611816406s, input_fired_time: 17.5296382904, output_fired_time: 8.7578859329, propagate_forward_time: 424.8605041504, propagate_backward_time: 654.2310791016, weight_update_time: 0.3150300384, other_time: 10.0820312500
epoch time: 88.9531936646s, input_fired_time: 1.4069052935, output_fired_time: 0.0000000000, propagate_forward_time: 85.7991485596, propagate_backward_time: 0.0000000000, weight_update_time: 0.0000000000, other_time: 1.7471389771
validation[          , genome  5894] predictions:    5885/  10000 (58.85%), best:    6339/10000 (63.39%), error:     13143.51074, best error:     10747.15625 on epoch:     0, epoch:   24/25, mu: 0.8854689598, learning_rate: 0.0000016716, weight_decay: 0.0000552328

epoch time: 1110.5208740234s, input_fired_time: 17.4312496185, output_fired_time: 8.6848096848, propagate_forward_time: 422.8227539063, propagate_backward_time: 651.6259155273, weight_update_time: 0.3198728561, other_time: 9.9561157227
epoch time: 88.2017288208s, input_fired_time: 1.3957268000, output_fired_time: 0.0000000000, propagate_forward_time: 85.0880126953, propagate_backward_time: 0.0000000000, weight_update_time: 0.0000000000, other_time: 1.7179870605
validation[          , genome  5894] predictions:    5815/  10000 (58.15%), best:    6339/10000 (63.39%), error:     13344.68359, best error:     10747.15625 on epoch:     0, epoch:   25/25, mu: 0.8920358419, learning_rate: 0.0000015515, weight_decay: 0.0000520156

evaluating best weights on full training data.
evaluting training set with running mean/variance:
epoch time: 503.7595214844s, input_fired_time: 8.2601766586, output_fired_time: 0.0000000000, propagate_forward_time: 485.4749450684, propagate_backward_time: 0.0000000000, weight_update_time: 0.0000000000, other_time: 10.0244140625
best training[          , genome  5894] predictions:   34427/  50000 (68.85%), best:    6339/10000 (63.39%), error:     47110.32031, best error:     10747.15625 on epoch:     0, epoch:   26/25, mu: 0.8981902003, learning_rate: 0.0000014401, weight_decay: 0.0000489857
evaluating best weights on test data.
evaluting test set with running mean/variance:
epoch time: 101.6319427490s, input_fired_time: 1.7155524492, output_fired_time: 0.0000000000, propagate_forward_time: 97.9657669067, propagate_backward_time: 0.0000000000, weight_update_time: 0.0000000000, other_time: 1.9506225586
   testing[          , genome  5894] predictions:    6339/  10000 (63.39%), best:    6339/10000 (63.39%), error:     10747.15625, best error:     10747.15625 on epoch:     0, epoch:   26/25, mu: 0.8981902003, learning_rate: 0.0000014401, weight_decay: 0.0000489857
backpropagation finished successfully!
00:10:35 (11136): called boinc_finish(0)

</stderr_txt>
]]>