Nameexact_genome_1512568488_42_6290_1
Workunit2602647
Created7 Dec 2017, 18:57:27 UTC
Sent7 Dec 2017, 19:01:08 UTC
Report deadline15 Dec 2017, 7:46:15 UTC
Received8 Dec 2017, 5:19:26 UTC
Server stateOver
OutcomeSuccess
Client stateDone
Exit status0 (0x0)
Computer ID66682
Run time3 hours 25 min 47 sec
CPU time3 hours 22 min 39 sec
Validate stateValid
Credit1,882.84
Device peak FLOPS4.55 GFLOPS
Application versionEXACT Batch Norm With Scaled FMP CNN Trainer v0.34
Peak working set size157.09 MB
Peak swap size160.02 MB
Peak disk usage2.27 MB

Stderr output

<core_client_version>7.8.3</core_client_version>
<![CDATA[
<stderr_txt>
  10000 (97.51%), best:    9734/10000 (97.34%), error:       858.28589, best error:       836.45258 on epoch:    13, epoch:   20/100, mu: 0.8143420219, learning_rate: 0.0044810725, weight_decay: 0.0001792429

epoch time: 102.4370651245s, input_fired_time: 1.6045128107, output_fired_time: 1.4314624071, propagate_forward_time: 38.9863281250, propagate_backward_time: 56.6236228943, weight_update_time: 0.2857791781, other_time: 3.7911338806
epoch time: 8.4388418198s, input_fired_time: 0.2001459450, output_fired_time: 0.0000000000, propagate_forward_time: 7.6646232605, propagate_backward_time: 0.0000000000, weight_update_time: 0.0000000000, other_time: 0.5740728378
validation[          , genome  6290] predictions:    9765/  10000 (97.65%), best:    9765/10000 (97.65%), error:       823.04431, best error:       823.04431 on epoch:    21, epoch:   21/100, mu: 0.8231249452, learning_rate: 0.0042570187, weight_decay: 0.0001702808

epoch time: 102.0634155273s, input_fired_time: 1.5980859995, output_fired_time: 1.4361280203, propagate_forward_time: 38.7411918640, propagate_backward_time: 56.3843231201, weight_update_time: 0.2915548384, other_time: 3.9036903381
epoch time: 8.5001230240s, input_fired_time: 0.2062822282, output_fired_time: 0.0000000000, propagate_forward_time: 7.7263660431, propagate_backward_time: 0.0000000000, weight_update_time: 0.0000000000, other_time: 0.5674743652
validation[          , genome  6290] predictions:    9741/  10000 (97.41%), best:    9765/10000 (97.65%), error:       827.45703, best error:       823.04431 on epoch:    21, epoch:   22/100, mu: 0.8314687014, learning_rate: 0.0040441677, weight_decay: 0.0001617667

epoch time: 101.5849990845s, input_fired_time: 1.5992227793, output_fired_time: 1.4296872616, propagate_forward_time: 38.4919052124, propagate_backward_time: 56.0512466431, weight_update_time: 0.2972607613, other_time: 4.0129394531
epoch time: 8.2405433655s, input_fired_time: 0.2035804242, output_fired_time: 0.0000000000, propagate_forward_time: 7.4539732933, propagate_backward_time: 0.0000000000, weight_update_time: 0.0000000000, other_time: 0.5829892159
validation[          , genome  6290] predictions:    9728/  10000 (97.28%), best:    9765/10000 (97.65%), error:       893.42188, best error:       823.04431 on epoch:    21, epoch:   23/100, mu: 0.8393952847, learning_rate: 0.0038419592, weight_decay: 0.0001536784

epoch time: 100.4803619385s, input_fired_time: 1.5876548290, output_fired_time: 1.4233903885, propagate_forward_time: 38.0392837524, propagate_backward_time: 55.5302886963, weight_update_time: 0.2916762829, other_time: 3.8997421265
epoch time: 8.4141187668s, input_fired_time: 0.2062346786, output_fired_time: 0.0000000000, propagate_forward_time: 7.6579446793, propagate_backward_time: 0.0000000000, weight_update_time: 0.0000000000, other_time: 0.5499391556
validation[          , genome  6290] predictions:    9704/  10000 (97.04%), best:    9765/10000 (97.65%), error:       918.89221, best error:       823.04431 on epoch:    21, epoch:   24/100, mu: 0.8469254971, learning_rate: 0.0036498611, weight_decay: 0.0001459945

arguments:
	'projects/csgrid.org_csg/exact_client_0.34_windows_x86_64.exe'
	'--training_file'
	'training_samples.bin'
	'--validation_file'
	'validation_samples.bin'
	'--testing_file'
	'testing_samples.bin'
	'--genome_file'
	'input_genome.txt'
	'--output_file'
	'output_genome.txt'
	'--checkpoint_file'
	'checkpoint.txt'
converting arguments to vector
boincified training filename: '../../projects/csgrid.org_csg/mnist_training_50k.bin'
boincified validation filename: '../../projects/csgrid.org_csg/mnist_validation_10k.bin'
boincified testing filename: '../../projects/csgrid.org_csg/mnist_testing_new.bin'
boincified genome filename: '../../projects/csgrid.org_csg/exact_genome_1512568488_42_6290.txt'
boincified output filename: '../../projects/csgrid.org_csg/exact_genome_1512568488_42_6290_1_r1569039062_0'
boincified checkpoint filename: 'checkpoint.txt'
loading genome
starting from checkpoint file: 'checkpoint.txt'
read CNN_Genome file with version string: 'v0.33'
read exact_id: -1
read genome_id: -1
read initial_mu: 0.5
read mu: 0.854079
read mu_delta: 0.95
read initial_learning_rate: 0.0125
read learning_rate: 0.00346737
read learning_rate_delta: 0.95
read initial_weight_decay: 0.0005
read weight_decay: 0.000138695
read weight_decay_delta: 0.95
read batch_size: 50
read epsilon: 1e-07
read alpha: 0.1
read input_dropout_probability: 0
read hidden_dropout_probability: 0
read velocity_reset: 1000
read epoch: 25
read max_epochs: 100
read reset_weights: 0
read padding: 2
read best_epoch: 21
read number_validation_images: 10000
read best_validation_predictions: 9765
read best_validation_error: 823.044
read number_training_images: 50000
read training_predictions: 0
read training_error: 1e+07
read number_test_images: 10000
read test_predictions: 0
read test_error: 1e+07
read generation_id: 6290
read normal distribution: '1 0 0'
generator_str: '1330067157'
read generator: 1330067157
read generated_by_map:
1 crossover 1
reading 23 nodes.
reading 62 edges.
number input nodes: 1
number softmax nodes: 10
order_size: 50000
parsed input file
loaded genome
loading images
number_classes: 10
channels: 1
width: 28
height: 28
reading image set with 4923 images.
reading image set with 5742 images.
reading image set with 4958 images.
reading image set with 5131 images.
reading image set with 4842 images.
reading image set with 4421 images.
reading image set with 4918 images.
reading image set with 5265 images.
reading image set with 4851 images.
reading image set with 4949 images.
image_size: 1x28x28 = 784
read 50000 images.
    class    0: 4923
    class    1: 5742
    class    2: 4958
    class    3: 5131
    class    4: 4842
    class    5: 4421
    class    6: 4918
    class    7: 5265
    class    8: 4851
    class    9: 4949
calculating averages and standard deviations for images
average pixel value for channel 0: 0.130530551075935
pixel variance for channel 0: 0.0948646068572998
pixel standard deviation for channel 0: 0.308000981807709
number_classes: 10
channels: 1
width: 28
height: 28
reading image set with 1000 images.
reading image set with 1000 images.
reading image set with 1000 images.
reading image set with 1000 images.
reading image set with 1000 images.
reading image set with 1000 images.
reading image set with 1000 images.
reading image set with 1000 images.
reading image set with 1000 images.
reading image set with 1000 images.
image_size: 1x28x28 = 784
read 10000 images.
    class    0: 1000
    class    1: 1000
    class    2: 1000
    class    3: 1000
    class    4: 1000
    class    5: 1000
    class    6: 1000
    class    7: 1000
    class    8: 1000
    class    9: 1000
number_classes: 10
channels: 1
width: 28
height: 28
reading image set with 980 images.
reading image set with 1135 images.
reading image set with 1032 images.
reading image set with 1010 images.
reading image set with 982 images.
reading image set with 892 images.
reading image set with 958 images.
reading image set with 1028 images.
reading image set with 974 images.
reading image set with 1009 images.
image_size: 1x28x28 = 784
read 10000 images.
    class    0: 980
    class    1: 1135
    class    2: 1032
    class    3: 1010
    class    4: 982
    class    5: 892
    class    6: 958
    class    7: 1028
    class    8: 974
    class    9: 1009
loaded images
starting backpropagation!
epoch time: 104.724731445313s, input_fired_time: 2.13325476646423, output_fired_time: 1.99699091911316, propagate_forward_time: 38.254451751709, propagate_backward_time: 56.4013175964355, weight_update_time: 0.427965581417084, other_time: 5.93871307373047
epoch time: 8.42711925506592s, input_fired_time: 0.285313159227371, output_fired_time: 0, propagate_forward_time: 7.32951307296753, propagate_backward_time: 0, weight_update_time: 0, other_time: 0.812292575836182
validation[          , genome  6290] predictions:    9304/  10000 (93.04%), best:    9765/10000 (97.65%), error:      2360.89795, best error:       823.04431 on epoch:    21, epoch:   25/100, mu: 0.8540792465, learning_rate: 0.0034673680, weight_decay: 0.0001386948

resetting weights to those on epoch: 21
epoch time: 104.9142837524s, input_fired_time: 2.1761524677, output_fired_time: 2.0431549549, propagate_forward_time: 38.3973159790, propagate_backward_time: 56.4128952026, weight_update_time: 0.4122890234, other_time: 5.8847656250
epoch time: 9.1604862213s, input_fired_time: 0.3270641565, output_fired_time: 0.0000000000, propagate_forward_time: 7.8921384811, propagate_backward_time: 0.0000000000, weight_update_time: 0.0000000000, other_time: 0.9412832260
validation[          , genome  6290] predictions:    9765/  10000 (97.65%), best:    9765/10000 (97.65%), error:       760.74731, best error:       760.74731 on epoch:    26, epoch:   26/100, mu: 0.8608753085, learning_rate: 0.0032939995, weight_decay: 0.0001317600

epoch time: 103.6885681152s, input_fired_time: 2.0887525082, output_fired_time: 1.9779560566, propagate_forward_time: 37.8568344116, propagate_backward_time: 55.9997444153, weight_update_time: 0.4120344222, other_time: 5.7652778625
epoch time: 8.8579616547s, input_fired_time: 0.3114653528, output_fired_time: 0.0000000000, propagate_forward_time: 7.6630988121, propagate_backward_time: 0.0000000000, weight_update_time: 0.0000000000, other_time: 0.8833975792
validation[          , genome  6290] predictions:    9706/  10000 (97.06%), best:    9765/10000 (97.65%), error:       964.82349, best error:       760.74731 on epoch:    26, epoch:   27/100, mu: 0.8673315644, learning_rate: 0.0031292995, weight_decay: 0.0001251720

epoch time: 104.7081909180s, input_fired_time: 2.1390848160, output_fired_time: 2.0364730358, propagate_forward_time: 38.2236595154, propagate_backward_time: 56.3768844604, weight_update_time: 0.4213916957, other_time: 5.9320869446
epoch time: 8.6827154160s, input_fired_time: 0.3219645321, output_fired_time: 0.0000000000, propagate_forward_time: 7.4375948906, propagate_backward_time: 0.0000000000, weight_update_time: 0.0000000000, other_time: 0.9231562614
validation[          , genome  6290] predictions:    9752/  10000 (97.52%), best:    9765/10000 (97.65%), error:       825.90247, best error:       760.74731 on epoch:    26, epoch:   28/100, mu: 0.8734650016, learning_rate: 0.0029728345, weight_decay: 0.0001189134

epoch time: 106.5209732056s, input_fired_time: 2.1363334656, output_fired_time: 2.0386953354, propagate_forward_time: 38.9386672974, propagate_backward_time: 57.4176788330, weight_update_time: 0.4270837009, other_time: 5.9896011353
epoch time: 10.4194097519s, input_fired_time: 0.4089325666, output_fired_time: 0.0000000000, propagate_forward_time: 8.8364877701, propagate_backward_time: 0.0000000000, weight_update_time: 0.0000000000, other_time: 1.1739892960
validation[          , genome  6290] predictions:    9709/  10000 (97.09%), best:    9765/10000 (97.65%), error:       982.08893, best error:       760.74731 on epoch:    26, epoch:   29/100, mu: 0.8792917728, learning_rate: 0.0028241926, weight_decay: 0.0001129677

epoch time: 120.8897323608s, input_fired_time: 2.8443827629, output_fired_time: 2.7553238869, propagate_forward_time: 43.6469306946, propagate_backward_time: 63.9392242432, weight_update_time: 0.5407207012, other_time: 7.7038726807
epoch time: 10.0082473755s, input_fired_time: 0.3550847173, output_fired_time: 0.0000000000, propagate_forward_time: 8.5868158340, propagate_backward_time: 0.0000000000, weight_update_time: 0.0000000000, other_time: 1.0663471222
validation[          , genome  6290] predictions:    9778/  10000 (97.78%), best:    9778/10000 (97.78%), error:       737.13416, best error:       737.13416 on epoch:    30, epoch:   30/100, mu: 0.8848271966, learning_rate: 0.0026829829, weight_decay: 0.0001073193

epoch time: 131.5350189209s, input_fired_time: 3.3900885582, output_fired_time: 3.1501824856, propagate_forward_time: 47.3091926575, propagate_backward_time: 68.9111404419, weight_update_time: 0.5792946219, other_time: 8.7744064331
epoch time: 9.7813072205s, input_fired_time: 0.3240153193, output_fired_time: 0.0000000000, propagate_forward_time: 8.4651231766, propagate_backward_time: 0.0000000000, weight_update_time: 0.0000000000, other_time: 0.9921684265
validation[          , genome  6290] predictions:    9721/  10000 (97.21%), best:    9778/10000 (97.78%), error:       880.21387, best error:       737.13416 on epoch:    30, epoch:   31/100, mu: 0.8900858164, learning_rate: 0.0025488338, weight_decay: 0.0001019534

epoch time: 122.3081283569s, input_fired_time: 2.9384732246, output_fired_time: 2.8639140129, propagate_forward_time: 44.0393981934, propagate_backward_time: 64.5218582153, weight_update_time: 0.5406041145, other_time: 7.9444808960
epoch time: 10.1331415176s, input_fired_time: 0.3782982230, output_fired_time: 0.0000000000, propagate_forward_time: 8.6280040741, propagate_backward_time: 0.0000000000, weight_update_time: 0.0000000000, other_time: 1.1268396378
validation[          , genome  6290] predictions:    9696/  10000 (96.96%), best:    9778/10000 (97.78%), error:      1068.91797, best error:       737.13416 on epoch:    30, epoch:   32/100, mu: 0.8950815201, learning_rate: 0.0024213921, weight_decay: 0.0000968557

epoch time: 120.3326263428s, input_fired_time: 2.5478878021, output_fired_time: 2.4980494976, propagate_forward_time: 44.0206565857, propagate_backward_time: 64.1685180664, weight_update_time: 0.4819375277, other_time: 7.0975189209
epoch time: 9.5165004730s, input_fired_time: 0.2840873599, output_fired_time: 0.0000000000, propagate_forward_time: 8.3250923157, propagate_backward_time: 0.0000000000, weight_update_time: 0.0000000000, other_time: 0.9073209763
validation[          , genome  6290] predictions:    9749/  10000 (97.49%), best:    9778/10000 (97.78%), error:       855.45984, best error:       737.13416 on epoch:    30, epoch:   33/100, mu: 0.8998274207, learning_rate: 0.0023003225, weight_decay: 0.0000920129

epoch time: 121.0573196411s, input_fired_time: 2.7489759922, output_fired_time: 2.6945848465, propagate_forward_time: 43.8629951477, propagate_backward_time: 64.2506256104, weight_update_time: 0.5243217945, other_time: 7.5001373291
epoch time: 10.2922506332s, input_fired_time: 0.3693620861, output_fired_time: 0.0000000000, propagate_forward_time: 8.7720937729, propagate_backward_time: 0.0000000000, weight_update_time: 0.0000000000, other_time: 1.1507949829
validation[          , genome  6290] predictions:    9746/  10000 (97.46%), best:    9778/10000 (97.78%), error:       889.57910, best error:       737.13416 on epoch:    30, epoch:   34/100, mu: 0.9043360353, learning_rate: 0.0021853063, weight_decay: 0.0000874123

epoch time: 121.1283798218s, input_fired_time: 2.7347178459, output_fired_time: 2.6605544090, propagate_forward_time: 43.8214874268, propagate_backward_time: 64.3863525391, weight_update_time: 0.5243242383, other_time: 7.5252685547
epoch time: 10.0010042191s, input_fired_time: 0.3670105040, output_fired_time: 0.0000000000, propagate_forward_time: 8.5619401932, propagate_backward_time: 0.0000000000, weight_update_time: 0.0000000000, other_time: 1.0720539093
validation[          , genome  6290] predictions:    9785/  10000 (97.85%), best:    9778/10000 (97.78%), error:       753.57178, best error:       737.13416 on epoch:    30, epoch:   35/100, mu: 0.9086192250, learning_rate: 0.0020760410, weight_decay: 0.0000830417

epoch time: 122.0059738159s, input_fired_time: 2.8176851273, output_fired_time: 2.7987484932, propagate_forward_time: 43.9898910522, propagate_backward_time: 64.5369338989, weight_update_time: 0.5404688120, other_time: 7.8627090454
epoch time: 11.9517107010s, input_fired_time: 0.5028684735, output_fired_time: 0.0000000000, propagate_forward_time: 9.9378719330, propagate_backward_time: 0.0000000000, weight_update_time: 0.0000000000, other_time: 1.5109701157
validation[          , genome  6290] predictions:    9767/  10000 (97.67%), best:    9778/10000 (97.78%), error:       856.40778, best error:       737.13416 on epoch:    30, epoch:   36/100, mu: 0.9126882553, learning_rate: 0.0019722388, weight_decay: 0.0000788896

epoch time: 123.0062713623s, input_fired_time: 2.7713084221, output_fired_time: 2.6634206772, propagate_forward_time: 44.6502647400, propagate_backward_time: 65.3272781372, weight_update_time: 0.5127565861, other_time: 7.5940017700
epoch time: 10.8245506287s, input_fired_time: 0.4191036224, output_fired_time: 0.0000000000, propagate_forward_time: 9.1234264374, propagate_backward_time: 0.0000000000, weight_update_time: 0.0000000000, other_time: 1.2820205688
validation[          , genome  6290] predictions:    9719/  10000 (97.19%), best:    9778/10000 (97.78%), error:       970.26215, best error:       737.13416 on epoch:    30, epoch:   37/100, mu: 0.9165538549, learning_rate: 0.0018736268, weight_decay: 0.0000749451

epoch time: 122.1142807007s, input_fired_time: 2.7668743134, output_fired_time: 2.6546835899, propagate_forward_time: 44.1568870544, propagate_backward_time: 64.8874053955, weight_update_time: 0.5110731125, other_time: 7.6484222412
epoch time: 10.5520191193s, input_fired_time: 0.3482725918, output_fired_time: 0.0000000000, propagate_forward_time: 9.1064691544, propagate_backward_time: 0.0000000000, weight_update_time: 0.0000000000, other_time: 1.0972776413
validation[          , genome  6290] predictions:    9760/  10000 (97.60%), best:    9778/10000 (97.78%), error:       799.12457, best error:       737.13416 on epoch:    30, epoch:   38/100, mu: 0.9202261567, learning_rate: 0.0017799455, weight_decay: 0.0000711978

epoch time: 130.3276824951s, input_fired_time: 3.1668672562, output_fired_time: 3.0377342701, propagate_forward_time: 47.2035293579, propagate_backward_time: 68.5456161499, weight_update_time: 0.5444965363, other_time: 8.3739318848
epoch time: 10.1262502670s, input_fired_time: 0.3594474196, output_fired_time: 0.0000000000, propagate_forward_time: 8.6769704819, propagate_backward_time: 0.0000000000, weight_update_time: 0.0000000000, other_time: 1.0898323059
validation[          , genome  6290] predictions:    9789/  10000 (97.89%), best:    9778/10000 (97.78%), error:       745.70691, best error:       737.13416 on epoch:    30, epoch:   39/100, mu: 0.9237148762, learning_rate: 0.0016909481, weight_decay: 0.0000676379

epoch time: 121.7751235962s, input_fired_time: 2.7463412285, output_fired_time: 2.6555056572, propagate_forward_time: 44.3022842407, propagate_backward_time: 64.6170043945, weight_update_time: 0.5209109783, other_time: 7.4539947510
epoch time: 9.8453636169s, input_fired_time: 0.3407496512, output_fired_time: 0.0000000000, propagate_forward_time: 8.4647789001, propagate_backward_time: 0.0000000000, weight_update_time: 0.0000000000, other_time: 1.0398349762
validation[          , genome  6290] predictions:    9679/  10000 (96.79%), best:    9778/10000 (97.78%), error:      1031.18823, best error:       737.13416 on epoch:    30, epoch:   40/100, mu: 0.9270291328, learning_rate: 0.0016064007, weight_decay: 0.0000642560

epoch time: 120.4527359009s, input_fired_time: 2.6875336170, output_fired_time: 2.6948823929, propagate_forward_time: 43.6830177307, propagate_backward_time: 63.8340377808, weight_update_time: 0.5316976905, other_time: 7.5532608032
epoch time: 10.0804166794s, input_fired_time: 0.3949562013, output_fired_time: 0.0000000000, propagate_forward_time: 8.5350618362, propagate_backward_time: 0.0000000000, weight_update_time: 0.0000000000, other_time: 1.1503982544
validation[          , genome  6290] predictions:    9773/  10000 (97.73%), best:    9778/10000 (97.78%), error:       776.60925, best error:       737.13416 on epoch:    30, epoch:   41/100, mu: 0.9301776886, learning_rate: 0.0015260807, weight_decay: 0.0000610432

epoch time: 124.1742248535s, input_fired_time: 2.8914086819, output_fired_time: 2.8657166958, propagate_forward_time: 44.9176330566, propagate_backward_time: 65.6638183594, weight_update_time: 0.5301416516, other_time: 7.8356475830
epoch time: 10.2089347839s, input_fired_time: 0.3865827620, output_fired_time: 0.0000000000, propagate_forward_time: 8.7109851837, propagate_backward_time: 0.0000000000, weight_update_time: 0.0000000000, other_time: 1.1113672256
validation[          , genome  6290] predictions:    9737/  10000 (97.37%), best:    9778/10000 (97.78%), error:       908.73743, best error:       737.13416 on epoch:    30, epoch:   42/100, mu: 0.9331688285, learning_rate: 0.0014497766, weight_decay: 0.0000579911

epoch time: 121.0324401855s, input_fired_time: 2.8261234760, output_fired_time: 2.7787790298, propagate_forward_time: 43.7197875977, propagate_backward_time: 63.8743743896, weight_update_time: 0.5499957204, other_time: 7.8333740234
epoch time: 13.2352304459s, input_fired_time: 0.5518539548, output_fired_time: 0.0000000000, propagate_forward_time: 10.9675149918, propagate_backward_time: 0.0000000000, weight_update_time: 0.0000000000, other_time: 1.7158613205
validation[          , genome  6290] predictions:    9774/  10000 (97.74%), best:    9778/10000 (97.78%), error:       759.59064, best error:       737.13416 on epoch:    30, epoch:   43/100, mu: 0.9360103607, learning_rate: 0.0013772878, weight_decay: 0.0000550915

epoch time: 123.3778762817s, input_fired_time: 2.7927989960, output_fired_time: 2.7660453320, propagate_forward_time: 44.8286781311, propagate_backward_time: 65.3326034546, weight_update_time: 0.5105734468, other_time: 7.6577529907
epoch time: 10.2750310898s, input_fired_time: 0.3963687420, output_fired_time: 0.0000000000, propagate_forward_time: 8.7225809097, propagate_backward_time: 0.0000000000, weight_update_time: 0.0000000000, other_time: 1.1560811996
validation[          , genome  6290] predictions:    9732/  10000 (97.32%), best:    9778/10000 (97.78%), error:       848.23346, best error:       737.13416 on epoch:    30, epoch:   44/100, mu: 0.9387098551, learning_rate: 0.0013084234, weight_decay: 0.0000523369

epoch time: 122.8602600098s, input_fired_time: 2.7967722416, output_fired_time: 2.7460353374, propagate_forward_time: 44.5320281982, propagate_backward_time: 65.1001968384, weight_update_time: 0.5267555714, other_time: 7.6852264404
epoch time: 10.3472566605s, input_fired_time: 0.4102831781, output_fired_time: 0.0000000000, propagate_forward_time: 8.7775650024, propagate_backward_time: 0.0000000000, weight_update_time: 0.0000000000, other_time: 1.1594085693
validation[          , genome  6290] predictions:    9764/  10000 (97.64%), best:    9778/10000 (97.78%), error:       795.41406, best error:       737.13416 on epoch:    30, epoch:   45/100, mu: 0.9412743449, learning_rate: 0.0012430021, weight_decay: 0.0000497201

epoch time: 122.6907424927s, input_fired_time: 2.8397390842, output_fired_time: 2.8736810684, propagate_forward_time: 44.0586204529, propagate_backward_time: 64.7726440430, weight_update_time: 0.5595591068, other_time: 8.1460571289
epoch time: 9.8338537216s, input_fired_time: 0.3118215203, output_fired_time: 0.0000000000, propagate_forward_time: 8.5328330994, propagate_backward_time: 0.0000000000, weight_update_time: 0.0000000000, other_time: 0.9891986847
validation[          , genome  6290] predictions:    9759/  10000 (97.59%), best:    9778/10000 (97.78%), error:       788.95245, best error:       737.13416 on epoch:    30, epoch:   46/100, mu: 0.9437106252, learning_rate: 0.0011808521, weight_decay: 0.0000472341

epoch time: 121.5250854492s, input_fired_time: 2.6893992424, output_fired_time: 2.6519150734, propagate_forward_time: 44.0911865234, propagate_backward_time: 64.4758834839, weight_update_time: 0.5202448964, other_time: 7.6166992188
epoch time: 9.7880992889s, input_fired_time: 0.3078938723, output_fired_time: 0.0000000000, propagate_forward_time: 8.5022287369, propagate_backward_time: 0.0000000000, weight_update_time: 0.0000000000, other_time: 0.9779767990
validation[          , genome  6290] predictions:    9768/  10000 (97.68%), best:    9778/10000 (97.78%), error:       800.22943, best error:       737.13416 on epoch:    30, epoch:   47/100, mu: 0.9460250735, learning_rate: 0.0011218095, weight_decay: 0.0000448724

epoch time: 124.5679626465s, input_fired_time: 2.9702563286, output_fired_time: 2.9865205288, propagate_forward_time: 44.8323669434, propagate_backward_time: 65.5998535156, weight_update_time: 0.5584646463, other_time: 8.1789703369
epoch time: 10.1719217300s, input_fired_time: 0.3342997134, output_fired_time: 0.0000000000, propagate_forward_time: 8.7583150864, propagate_backward_time: 0.0000000000, weight_update_time: 0.0000000000, other_time: 1.0793066025
validation[          , genome  6290] predictions:    9755/  10000 (97.55%), best:    9778/10000 (97.78%), error:       861.78217, best error:       737.13416 on epoch:    30, epoch:   48/100, mu: 0.9482238293, learning_rate: 0.0010657189, weight_decay: 0.0000426288

epoch time: 122.2620239258s, input_fired_time: 2.7317366600, output_fired_time: 2.7103664875, propagate_forward_time: 44.3459625244, propagate_backward_time: 64.7630310059, weight_update_time: 0.5098724961, other_time: 7.7109298706
epoch time: 9.7797021866s, input_fired_time: 0.3047634661, output_fired_time: 0.0000000000, propagate_forward_time: 8.4899616241, propagate_backward_time: 0.0000000000, weight_update_time: 0.0000000000, other_time: 0.9849767685
validation[          , genome  6290] predictions:    9779/  10000 (97.79%), best:    9778/10000 (97.78%), error:       768.40045, best error:       737.13416 on epoch:    30, epoch:   49/100, mu: 0.9503126144, learning_rate: 0.0010124330, weight_decay: 0.0000404973

epoch time: 115.1821441650s, input_fired_time: 2.1865303516, output_fired_time: 2.0991768837, propagate_forward_time: 42.5717849731, propagate_backward_time: 62.1996040344, weight_update_time: 0.4019185603, other_time: 6.1250495911
epoch time: 9.8824405670s, input_fired_time: 0.3201305270, output_fired_time: 0.0000000000, propagate_forward_time: 8.5676813126, propagate_backward_time: 0.0000000000, weight_update_time: 0.0000000000, other_time: 0.9946289063
validation[          , genome  6290] predictions:    9752/  10000 (97.52%), best:    9778/10000 (97.78%), error:       821.18848, best error:       737.13416 on epoch:    30, epoch:   50/100, mu: 0.9522969723, learning_rate: 0.0009618113, weight_decay: 0.0000384725

epoch time: 115.8303833008s, input_fired_time: 2.2784092426, output_fired_time: 2.2201251984, propagate_forward_time: 42.5426635742, propagate_backward_time: 62.4598464966, weight_update_time: 0.4190932810, other_time: 6.3293380737
epoch time: 10.5798635483s, input_fired_time: 0.3858421445, output_fired_time: 0.0000000000, propagate_forward_time: 9.0332107544, propagate_backward_time: 0.0000000000, weight_update_time: 0.0000000000, other_time: 1.1608104706
validation[          , genome  6290] predictions:    9750/  10000 (97.50%), best:    9778/10000 (97.78%), error:       834.82867, best error:       737.13416 on epoch:    30, epoch:   51/100, mu: 0.9541821480, learning_rate: 0.0009137207, weight_decay: 0.0000365488

epoch time: 121.1762084961s, input_fired_time: 2.5491268635, output_fired_time: 2.4694421291, propagate_forward_time: 44.2695999146, propagate_backward_time: 64.8058166504, weight_update_time: 0.4601600468, other_time: 7.0822219849
epoch time: 9.7449617386s, input_fired_time: 0.3045484126, output_fired_time: 0.0000000000, propagate_forward_time: 8.4874582291, propagate_backward_time: 0.0000000000, weight_update_time: 0.0000000000, other_time: 0.9529552460
validation[          , genome  6290] predictions:    9760/  10000 (97.60%), best:    9778/10000 (97.78%), error:       849.42755, best error:       737.13416 on epoch:    30, epoch:   52/100, mu: 0.9559730291, learning_rate: 0.0008680347, weight_decay: 0.0000347214

epoch time: 118.4783935547s, input_fired_time: 2.4313297272, output_fired_time: 2.3397300243, propagate_forward_time: 43.4739456177, propagate_backward_time: 63.5290489197, weight_update_time: 0.4529812038, other_time: 6.7043418884
epoch time: 9.6801834106s, input_fired_time: 0.3127604723, output_fired_time: 0.0000000000, propagate_forward_time: 8.4046497345, propagate_backward_time: 0.0000000000, weight_update_time: 0.0000000000, other_time: 0.9627733231
validation[          , genome  6290] predictions:    9747/  10000 (97.47%), best:    9778/10000 (97.78%), error:       818.82770, best error:       737.13416 on epoch:    30, epoch:   53/100, mu: 0.9576743841, learning_rate: 0.0008246329, weight_decay: 0.0000329853

epoch time: 119.3395156860s, input_fired_time: 2.5613596439, output_fired_time: 2.5080051422, propagate_forward_time: 43.4650115967, propagate_backward_time: 63.6208915710, weight_update_time: 0.4834791124, other_time: 7.1842460632
epoch time: 10.0377798080s, input_fired_time: 0.3328626752, output_fired_time: 0.0000000000, propagate_forward_time: 8.6693124771, propagate_backward_time: 0.0000000000, weight_update_time: 0.0000000000, other_time: 1.0356044769
validation[          , genome  6290] predictions:    9726/  10000 (97.26%), best:    9778/10000 (97.78%), error:       903.73224, best error:       737.13416 on epoch:    30, epoch:   54/100, mu: 0.9592906833, learning_rate: 0.0007834013, weight_decay: 0.0000313361

epoch time: 121.4384307861s, input_fired_time: 2.6706769466, output_fired_time: 2.5433442593, propagate_forward_time: 44.4082603455, propagate_backward_time: 64.6505432129, weight_update_time: 0.4681796432, other_time: 7.1656036377
epoch time: 9.8796815872s, input_fired_time: 0.3273910284, output_fired_time: 0.0000000000, propagate_forward_time: 8.5452499390, propagate_backward_time: 0.0000000000, weight_update_time: 0.0000000000, other_time: 1.0070409775
validation[          , genome  6290] predictions:    9730/  10000 (97.30%), best:    9778/10000 (97.78%), error:       866.78693, best error:       737.13416 on epoch:    30, epoch:   55/100, mu: 0.9608261585, learning_rate: 0.0007442312, weight_decay: 0.0000297693

epoch time: 115.9201660156s, input_fired_time: 2.3367166519, output_fired_time: 2.2432615757, propagate_forward_time: 42.4786338806, propagate_backward_time: 62.4077529907, weight_update_time: 0.4249733090, other_time: 6.4538040161
epoch time: 9.7952127457s, input_fired_time: 0.3164843023, output_fired_time: 0.0000000000, propagate_forward_time: 8.5402898788, propagate_backward_time: 0.0000000000, weight_update_time: 0.0000000000, other_time: 0.9384384155
validation[          , genome  6290] predictions:    9758/  10000 (97.58%), best:    9778/10000 (97.78%), error:       899.94025, best error:       737.13416 on epoch:    30, epoch:   56/100, mu: 0.9622848630, learning_rate: 0.0007070196, weight_decay: 0.0000282808

epoch time: 115.5971527100s, input_fired_time: 2.2141153812, output_fired_time: 2.0851366520, propagate_forward_time: 42.6533622742, propagate_backward_time: 62.3522758484, weight_update_time: 0.4011478424, other_time: 6.2922554016
epoch time: 9.6226511002s, input_fired_time: 0.3045641780, output_fired_time: 0.0000000000, propagate_forward_time: 8.3425292969, propagate_backward_time: 0.0000000000, weight_update_time: 0.0000000000, other_time: 0.9755573273
validation[          , genome  6290] predictions:    9767/  10000 (97.67%), best:    9778/10000 (97.78%), error:       781.62762, best error:       737.13416 on epoch:    30, epoch:   57/100, mu: 0.9636706114, learning_rate: 0.0006716687, weight_decay: 0.0000268668

epoch time: 115.9568481445s, input_fired_time: 2.2588114738, output_fired_time: 2.1779072285, propagate_forward_time: 42.6848907471, propagate_backward_time: 62.4924964905, weight_update_time: 0.4175133407, other_time: 6.3427391052
epoch time: 9.6039800644s, input_fired_time: 0.3033691943, output_fired_time: 0.0000000000, propagate_forward_time: 8.3790702820, propagate_backward_time: 0.0000000000, weight_update_time: 0.0000000000, other_time: 0.9215402603
validation[          , genome  6290] predictions:    9780/  10000 (97.80%), best:    9780/10000 (97.80%), error:       730.10120, best error:       730.10120 on epoch:    58, epoch:   58/100, mu: 0.9649870992, learning_rate: 0.0006380852, weight_decay: 0.0000255234

epoch time: 114.7420425415s, input_fired_time: 2.1276903152, output_fired_time: 2.0082366467, propagate_forward_time: 42.4959983826, propagate_backward_time: 62.0644111633, weight_update_time: 0.3938621879, other_time: 6.0456962585
epoch time: 9.3598747253s, input_fired_time: 0.2957828641, output_fired_time: 0.0000000000, propagate_forward_time: 8.1431102753, propagate_backward_time: 0.0000000000, weight_update_time: 0.0000000000, other_time: 0.9209814072
validation[          , genome  6290] predictions:    9779/  10000 (97.79%), best:    9780/10000 (97.80%), error:       803.35699, best error:       730.10120 on epoch:    58, epoch:   59/100, mu: 0.9662377238, learning_rate: 0.0006061810, weight_decay: 0.0000242472

epoch time: 113.8680496216s, input_fired_time: 2.0774703026, output_fired_time: 1.9669811726, propagate_forward_time: 42.3350563049, propagate_backward_time: 61.7889251709, weight_update_time: 0.3710862994, other_time: 5.6996154785
epoch time: 9.1875867844s, input_fired_time: 0.2462328970, output_fired_time: 0.0000000000, propagate_forward_time: 8.1700153351, propagate_backward_time: 0.0000000000, weight_update_time: 0.0000000000, other_time: 0.7713384628
validation[          , genome  6290] predictions:    9785/  10000 (97.85%), best:    9780/10000 (97.80%), error:       731.30121, best error:       730.10120 on epoch:    58, epoch:   60/100, mu: 0.9674258232, learning_rate: 0.0005758719, weight_decay: 0.0000230349

epoch time: 110.2629776001s, input_fired_time: 1.8908034563, output_fired_time: 1.7803293467, propagate_forward_time: 41.1531181335, propagate_backward_time: 60.2371330261, weight_update_time: 0.3451645970, other_time: 5.2016029358
epoch time: 9.2927293777s, input_fired_time: 0.2468262166, output_fired_time: 0.0000000000, propagate_forward_time: 8.2688226700, propagate_backward_time: 0.0000000000, weight_update_time: 0.0000000000, other_time: 0.7770805359
validation[          , genome  6290] predictions:    9764/  10000 (97.64%), best:    9780/10000 (97.80%), error:       789.92297, best error:       730.10120 on epoch:    58, epoch:   61/100, mu: 0.9685545564, learning_rate: 0.0005470783, weight_decay: 0.0000218831

epoch time: 112.0557250977s, input_fired_time: 1.9279606342, output_fired_time: 1.8192176819, propagate_forward_time: 41.9249725342, propagate_backward_time: 61.1209640503, weight_update_time: 0.3515638113, other_time: 5.2626113892
epoch time: 9.2915802002s, input_fired_time: 0.2397730201, output_fired_time: 0.0000000000, propagate_forward_time: 8.2898473740, propagate_backward_time: 0.0000000000, weight_update_time: 0.0000000000, other_time: 0.7619600296
validation[          , genome  6290] predictions:    9768/  10000 (97.68%), best:    9780/10000 (97.80%), error:       798.00989, best error:       730.10120 on epoch:    58, epoch:   62/100, mu: 0.9696268439, learning_rate: 0.0005197244, weight_decay: 0.0000207890

epoch time: 112.2719268799s, input_fired_time: 1.9328175783, output_fired_time: 1.8002085686, propagate_forward_time: 41.8868331909, propagate_backward_time: 61.3804054260, weight_update_time: 0.3430978358, other_time: 5.2716636658
epoch time: 9.1409044266s, input_fired_time: 0.2546549439, output_fired_time: 0.0000000000, propagate_forward_time: 8.1109733582, propagate_backward_time: 0.0000000000, weight_update_time: 0.0000000000, other_time: 0.7752761841
validation[          , genome  6290] predictions:    9732/  10000 (97.32%), best:    9780/10000 (97.80%), error:       896.07825, best error:       730.10120 on epoch:    58, epoch:   63/100, mu: 0.9706454873, learning_rate: 0.0004937382, weight_decay: 0.0000197495

epoch time: 112.3985061646s, input_fired_time: 1.9310281277, output_fired_time: 1.7944850922, propagate_forward_time: 41.9664535522, propagate_backward_time: 61.4941520691, weight_update_time: 0.3468267620, other_time: 5.2123832703
epoch time: 9.3613138199s, input_fired_time: 0.2481735200, output_fired_time: 0.0000000000, propagate_forward_time: 8.3182897568, propagate_backward_time: 0.0000000000, weight_update_time: 0.0000000000, other_time: 0.7948503494
validation[          , genome  6290] predictions:    9790/  10000 (97.90%), best:    9780/10000 (97.80%), error:       745.58734, best error:       730.10120 on epoch:    58, epoch:   64/100, mu: 0.9716132283, learning_rate: 0.0004690512, weight_decay: 0.0000187621

epoch time: 112.2717437744s, input_fired_time: 1.9256860018, output_fired_time: 1.7806314230, propagate_forward_time: 42.0222625732, propagate_backward_time: 61.3219108582, weight_update_time: 0.3514027596, other_time: 5.2212486267
epoch time: 9.4158287048s, input_fired_time: 0.2471512407, output_fired_time: 0.0000000000, propagate_forward_time: 8.3832168579, propagate_backward_time: 0.0000000000, weight_update_time: 0.0000000000, other_time: 0.7854604721
validation[          , genome  6290] predictions:    9741/  10000 (97.41%), best:    9780/10000 (97.80%), error:       894.12543, best error:       730.10120 on epoch:    58, epoch:   65/100, mu: 0.9725325704, learning_rate: 0.0004455987, weight_decay: 0.0000178240

epoch time: 111.3948898315s, input_fired_time: 1.8650646210, output_fired_time: 1.7204821110, propagate_forward_time: 41.7801818848, propagate_backward_time: 61.0303649902, weight_update_time: 0.3317571282, other_time: 4.9987945557
epoch time: 9.5621576309s, input_fired_time: 0.2609866261, output_fired_time: 0.0000000000, propagate_forward_time: 8.5012168884, propagate_backward_time: 0.0000000000, weight_update_time: 0.0000000000, other_time: 0.7999544144
validation[          , genome  6290] predictions:    9767/  10000 (97.67%), best:    9780/10000 (97.80%), error:       734.54907, best error:       730.10120 on epoch:    58, epoch:   66/100, mu: 0.9734059572, learning_rate: 0.0004233187, weight_decay: 0.0000169328

epoch time: 110.4538879395s, input_fired_time: 1.8484367132, output_fired_time: 1.7170811892, propagate_forward_time: 41.4316291809, propagate_backward_time: 60.5098686218, weight_update_time: 0.3289346099, other_time: 4.9468727112
epoch time: 9.2788562775s, input_fired_time: 0.2392515093, output_fired_time: 0.0000000000, propagate_forward_time: 8.2947244644, propagate_backward_time: 0.0000000000, weight_update_time: 0.0000000000, other_time: 0.7448806763
validation[          , genome  6290] predictions:    9772/  10000 (97.72%), best:    9780/10000 (97.80%), error:       754.85114, best error:       730.10120 on epoch:    58, epoch:   67/100, mu: 0.9742356539, learning_rate: 0.0004021528, weight_decay: 0.0000160861

epoch time: 112.1204299927s, input_fired_time: 1.8701651096, output_fired_time: 1.7395716906, propagate_forward_time: 42.0264816284, propagate_backward_time: 61.4352188110, weight_update_time: 0.3406000435, other_time: 5.0489959717
epoch time: 9.2205104828s, input_fired_time: 0.2495699078, output_fired_time: 0.0000000000, propagate_forward_time: 8.2532501221, propagate_backward_time: 0.0000000000, weight_update_time: 0.0000000000, other_time: 0.7176904678
validation[          , genome  6290] predictions:    9765/  10000 (97.65%), best:    9780/10000 (97.80%), error:       805.78589, best error:       730.10120 on epoch:    58, epoch:   68/100, mu: 0.9750238657, learning_rate: 0.0003820452, weight_decay: 0.0000152818

epoch time: 111.5763549805s, input_fired_time: 1.8976674080, output_fired_time: 1.7417370081, propagate_forward_time: 41.8888664246, propagate_backward_time: 61.0870323181, weight_update_time: 0.3299903274, other_time: 4.9610481262
epoch time: 9.3220758438s, input_fired_time: 0.2626182437, output_fired_time: 0.0000000000, propagate_forward_time: 8.2802209854, propagate_backward_time: 0.0000000000, weight_update_time: 0.0000000000, other_time: 0.7792367935
validation[          , genome  6290] predictions:    9662/  10000 (96.62%), best:    9780/10000 (97.80%), error:      1159.92004, best error:       730.10120 on epoch:    58, epoch:   69/100, mu: 0.9757726789, learning_rate: 0.0003629429, weight_decay: 0.0000145177

epoch time: 110.6628799438s, input_fired_time: 1.8382421732, output_fired_time: 1.7107484341, propagate_forward_time: 41.5974082947, propagate_backward_time: 60.5791549683, weight_update_time: 0.3290221989, other_time: 4.9373245239
epoch time: 9.0969505310s, input_fired_time: 0.2308910787, output_fired_time: 0.0000000000, propagate_forward_time: 8.1477289200, propagate_backward_time: 0.0000000000, weight_update_time: 0.0000000000, other_time: 0.7183303833
validation[          , genome  6290] predictions:    9756/  10000 (97.56%), best:    9780/10000 (97.80%), error:       828.94550, best error:       730.10120 on epoch:    58, epoch:   70/100, mu: 0.9764840603, learning_rate: 0.0003447958, weight_decay: 0.0000137918

epoch time: 110.2216644287s, input_fired_time: 1.8746147156, output_fired_time: 1.7590501308, propagate_forward_time: 41.2605133057, propagate_backward_time: 60.3327064514, weight_update_time: 0.3327765763, other_time: 4.9947776794
epoch time: 9.4114408493s, input_fired_time: 0.2594534159, output_fired_time: 0.0000000000, propagate_forward_time: 8.3630523682, propagate_backward_time: 0.0000000000, weight_update_time: 0.0000000000, other_time: 0.7889347076
validation[          , genome  6290] predictions:    9743/  10000 (97.43%), best:    9780/10000 (97.80%), error:       929.93048, best error:       730.10120 on epoch:    58, epoch:   71/100, mu: 0.9771598577, learning_rate: 0.0003275560, weight_decay: 0.0000131022

epoch time: 110.5545883179s, input_fired_time: 1.8525247574, output_fired_time: 1.7279566526, propagate_forward_time: 41.4826316833, propagate_backward_time: 60.5243568420, weight_update_time: 0.3284507394, other_time: 4.9671134949
epoch time: 9.7815008163s, input_fired_time: 0.2621788681, output_fired_time: 0.0000000000, propagate_forward_time: 8.6824598312, propagate_backward_time: 0.0000000000, weight_update_time: 0.0000000000, other_time: 0.8368625641
validation[          , genome  6290] predictions:    9753/  10000 (97.53%), best:    9780/10000 (97.80%), error:       847.55408, best error:       730.10120 on epoch:    58, epoch:   72/100, mu: 0.9778018594, learning_rate: 0.0003111782, weight_decay: 0.0000124471

epoch time: 111.5344772339s, input_fired_time: 1.8613344431, output_fired_time: 1.7121701241, propagate_forward_time: 41.8984146118, propagate_backward_time: 61.0885963440, weight_update_time: 0.3269770741, other_time: 4.9739570618
epoch time: 9.1585521698s, input_fired_time: 0.2417345345, output_fired_time: 0.0000000000, propagate_forward_time: 8.1676073074, propagate_backward_time: 0.0000000000, weight_update_time: 0.0000000000, other_time: 0.7492103577
validation[          , genome  6290] predictions:    9775/  10000 (97.75%), best:    9780/10000 (97.80%), error:       784.97955, best error:       730.10120 on epoch:    58, epoch:   73/100, mu: 0.9784117937, learning_rate: 0.0002956192, weight_decay: 0.0000118248

epoch time: 109.6491317749s, input_fired_time: 1.8074012995, output_fired_time: 1.6602901220, propagate_forward_time: 41.1391906738, propagate_backward_time: 60.2792472839, weight_update_time: 0.3191201985, other_time: 4.7629966736
epoch time: 9.2615575790s, input_fired_time: 0.2403932363, output_fired_time: 0.0000000000, propagate_forward_time: 8.2897434235, propagate_backward_time: 0.0000000000, weight_update_time: 0.0000000000, other_time: 0.7314205170
validation[          , genome  6290] predictions:    9797/  10000 (97.97%), best:    9797/10000 (97.97%), error:       697.63519, best error:       697.63519 on epoch:    74, epoch:   74/100, mu: 0.9789912105, learning_rate: 0.0002808383, weight_decay: 0.0000112335

epoch time: 110.3407974243s, input_fired_time: 1.7982013226, output_fired_time: 1.6672862768, propagate_forward_time: 41.4610671997, propagate_backward_time: 60.5907211304, weight_update_time: 0.3155861795, other_time: 4.8235168457
epoch time: 9.1990299225s, input_fired_time: 0.2343143821, output_fired_time: 0.0000000000, propagate_forward_time: 8.2414016724, propagate_backward_time: 0.0000000000, weight_update_time: 0.0000000000, other_time: 0.7233142853
validation[          , genome  6290] predictions:    9748/  10000 (97.48%), best:    9797/10000 (97.97%), error:       809.41394, best error:       697.63519 on epoch:    74, epoch:   75/100, mu: 0.9795416594, learning_rate: 0.0002667964, weight_decay: 0.0000106719

epoch time: 109.6575927734s, input_fired_time: 1.8102487326, output_fired_time: 1.6836552620, propagate_forward_time: 41.1735534668, propagate_backward_time: 60.1382827759, weight_update_time: 0.3211670518, other_time: 4.8518524170
epoch time: 9.0247640610s, input_fired_time: 0.2365575582, output_fired_time: 0.0000000000, propagate_forward_time: 8.0556697845, propagate_backward_time: 0.0000000000, weight_update_time: 0.0000000000, other_time: 0.7325363159
validation[          , genome  6290] predictions:    9795/  10000 (97.95%), best:    9797/10000 (97.97%), error:       759.74109, best error:       697.63519 on epoch:    74, epoch:   76/100, mu: 0.9800645709, learning_rate: 0.0002534565, weight_decay: 0.0000101383

epoch time: 109.8842239380s, input_fired_time: 1.8264325857, output_fired_time: 1.6890363693, propagate_forward_time: 41.2186698914, propagate_backward_time: 60.2114906311, weight_update_time: 0.3230928481, other_time: 4.9385948181
epoch time: 9.2882757187s, input_fired_time: 0.2514880300, output_fired_time: 0.0000000000, propagate_forward_time: 8.2593088150, propagate_backward_time: 0.0000000000, weight_update_time: 0.0000000000, other_time: 0.7774791718
validation[          , genome  6290] predictions:    9785/  10000 (97.85%), best:    9797/10000 (97.97%), error:       750.66931, best error:       697.63519 on epoch:    74, epoch:   77/100, mu: 0.9805613160, learning_rate: 0.0002407837, weight_decay: 0.0000096314

epoch time: 111.0475311279s, input_fired_time: 1.8810726404, output_fired_time: 1.7679446936, propagate_forward_time: 41.5284957886, propagate_backward_time: 60.8154869080, weight_update_time: 0.3284474611, other_time: 5.0545310974
epoch time: 9.3995952606s, input_fired_time: 0.2539826632, output_fired_time: 0.0000000000, propagate_forward_time: 8.3363285065, propagate_backward_time: 0.0000000000, weight_update_time: 0.0000000000, other_time: 0.8092842102
validation[          , genome  6290] predictions:    9791/  10000 (97.91%), best:    9797/10000 (97.97%), error:       701.22534, best error:       697.63519 on epoch:    74, epoch:   78/100, mu: 0.9810332656, learning_rate: 0.0002287445, weight_decay: 0.0000091498

epoch time: 111.0772552490s, input_fired_time: 1.8933339119, output_fired_time: 1.7201091051, propagate_forward_time: 41.5032043457, propagate_backward_time: 60.8778915405, weight_update_time: 0.3310564160, other_time: 5.0827178955
epoch time: 9.3675413132s, input_fired_time: 0.2508894205, output_fired_time: 0.0000000000, propagate_forward_time: 8.3037958145, propagate_backward_time: 0.0000000000, weight_update_time: 0.0000000000, other_time: 0.8128557205
validation[          , genome  6290] predictions:    9748/  10000 (97.48%), best:    9797/10000 (97.97%), error:       816.32697, best error:       697.63519 on epoch:    74, epoch:   79/100, mu: 0.9814816117, learning_rate: 0.0002173073, weight_decay: 0.0000086923

epoch time: 110.8098678589s, input_fired_time: 1.8587285280, output_fired_time: 1.7297824621, propagate_forward_time: 41.5001449585, propagate_backward_time: 60.6391639709, weight_update_time: 0.3284608722, other_time: 5.0820503235
epoch time: 9.1602230072s, input_fired_time: 0.2410897911, output_fired_time: 0.0000000000, propagate_forward_time: 8.1732368469, propagate_backward_time: 0.0000000000, weight_update_time: 0.0000000000, other_time: 0.7458963394
validation[          , genome  6290] predictions:    9754/  10000 (97.54%), best:    9797/10000 (97.97%), error:       846.21423, best error:       697.63519 on epoch:    74, epoch:   80/100, mu: 0.9819075465, learning_rate: 0.0002064419, weight_decay: 0.0000082577

epoch time: 111.9066162109s, input_fired_time: 1.9026445150, output_fired_time: 1.7591322660, propagate_forward_time: 41.8990173340, propagate_backward_time: 61.1905670166, weight_update_time: 0.3351586759, other_time: 5.1552581787
epoch time: 9.2527370453s, input_fired_time: 0.2390552610, output_fired_time: 0.0000000000, propagate_forward_time: 8.2471361160, propagate_backward_time: 0.0000000000, weight_update_time: 0.0000000000, other_time: 0.7665452957
validation[          , genome  6290] predictions:    9786/  10000 (97.86%), best:    9797/10000 (97.97%), error:       754.08624, best error:       697.63519 on epoch:    74, epoch:   81/100, mu: 0.9823121428, learning_rate: 0.0001961198, weight_decay: 0.0000078448

epoch time: 110.2555007935s, input_fired_time: 1.8437846899, output_fired_time: 1.7084800005, propagate_forward_time: 41.3575363159, propagate_backward_time: 60.3982162476, weight_update_time: 0.3238159418, other_time: 4.9474792480
epoch time: 9.0344467163s, input_fired_time: 0.2304700464, output_fired_time: 0.0000000000, propagate_forward_time: 8.0926837921, propagate_backward_time: 0.0000000000, weight_update_time: 0.0000000000, other_time: 0.7112932205
validation[          , genome  6290] predictions:    9734/  10000 (97.34%), best:    9797/10000 (97.97%), error:       884.97388, best error:       697.63519 on epoch:    74, epoch:   82/100, mu: 0.9826965332, learning_rate: 0.0001863138, weight_decay: 0.0000074526

epoch time: 111.9986343384s, input_fired_time: 1.8803241253, output_fired_time: 1.7692322731, propagate_forward_time: 41.9968376160, propagate_backward_time: 61.2550277710, weight_update_time: 0.3275286257, other_time: 5.0972061157
epoch time: 9.3047943115s, input_fired_time: 0.2408158779, output_fired_time: 0.0000000000, propagate_forward_time: 8.3105096817, propagate_backward_time: 0.0000000000, weight_update_time: 0.0000000000, other_time: 0.7534685135
validation[          , genome  6290] predictions:    9790/  10000 (97.90%), best:    9797/10000 (97.97%), error:       738.14948, best error:       697.63519 on epoch:    74, epoch:   83/100, mu: 0.9830617309, learning_rate: 0.0001769981, weight_decay: 0.0000070799

epoch time: 111.4254531860s, input_fired_time: 1.8852833509, output_fired_time: 1.7294943333, propagate_forward_time: 41.7818374634, propagate_backward_time: 60.9871711731, weight_update_time: 0.3241977096, other_time: 5.0416679382
epoch time: 9.2876234055s, input_fired_time: 0.2643108368, output_fired_time: 0.0000000000, propagate_forward_time: 8.2267503738, propagate_backward_time: 0.0000000000, weight_update_time: 0.0000000000, other_time: 0.7965621948
validation[          , genome  6290] predictions:    9774/  10000 (97.74%), best:    9797/10000 (97.97%), error:       810.00287, best error:       697.63519 on epoch:    74, epoch:   84/100, mu: 0.9834086299, learning_rate: 0.0001681482, weight_decay: 0.0000067259

epoch time: 111.0274963379s, input_fired_time: 1.9051674604, output_fired_time: 1.7727830410, propagate_forward_time: 41.5243568420, propagate_backward_time: 60.6936225891, weight_update_time: 0.3365487158, other_time: 5.1315727234
epoch time: 9.2899436951s, input_fired_time: 0.2444025278, output_fired_time: 0.0000000000, propagate_forward_time: 8.2602176666, propagate_backward_time: 0.0000000000, weight_update_time: 0.0000000000, other_time: 0.7853231430
validation[          , genome  6290] predictions:    9773/  10000 (97.73%), best:    9797/10000 (97.97%), error:       770.23492, best error:       697.63519 on epoch:    74, epoch:   85/100, mu: 0.9837381840, learning_rate: 0.0001597408, weight_decay: 0.0000063896

epoch time: 111.4516830444s, input_fired_time: 1.8969659805, output_fired_time: 1.7754417658, propagate_forward_time: 41.6463012695, propagate_backward_time: 61.0162582397, weight_update_time: 0.3311996162, other_time: 5.1167144775
epoch time: 9.3932609558s, input_fired_time: 0.2470002770, output_fired_time: 0.0000000000, propagate_forward_time: 8.3441553116, propagate_backward_time: 0.0000000000, weight_update_time: 0.0000000000, other_time: 0.8021049500
validation[          , genome  6290] predictions:    9763/  10000 (97.63%), best:    9797/10000 (97.97%), error:       842.70428, best error:       697.63519 on epoch:    74, epoch:   86/100, mu: 0.9840512872, learning_rate: 0.0001517538, weight_decay: 0.0000060702

epoch time: 111.6861114502s, input_fired_time: 1.8601326942, output_fired_time: 1.7356020212, propagate_forward_time: 41.7708549500, propagate_backward_time: 61.2725639343, weight_update_time: 0.3296023011, other_time: 5.0469551086
epoch time: 9.4246892929s, input_fired_time: 0.2456842810, output_fired_time: 0.0000000000, propagate_forward_time: 8.4217138290, propagate_backward_time: 0.0000000000, weight_update_time: 0.0000000000, other_time: 0.7572908401
validation[          , genome  6290] predictions:    9777/  10000 (97.77%), best:    9797/10000 (97.97%), error:       764.73676, best error:       697.63519 on epoch:    74, epoch:   87/100, mu: 0.9843487144, learning_rate: 0.0001441661, weight_decay: 0.0000057666

epoch time: 112.1940307617s, input_fired_time: 1.9100575447, output_fired_time: 1.7449804544, propagate_forward_time: 42.1227684021, propagate_backward_time: 61.3130912781, weight_update_time: 0.3344498277, other_time: 5.1031379700
epoch time: 9.5653886795s, input_fired_time: 0.2571223378, output_fired_time: 0.0000000000, propagate_forward_time: 8.5138120651, propagate_backward_time: 0.0000000000, weight_update_time: 0.0000000000, other_time: 0.7944545746
validation[          , genome  6290] predictions:    9773/  10000 (97.73%), best:    9797/10000 (97.97%), error:       771.67395, best error:       697.63519 on epoch:    74, epoch:   88/100, mu: 0.9846313000, learning_rate: 0.0001369578, weight_decay: 0.0000054783

epoch time: 112.6242599487s, input_fired_time: 1.9263370037, output_fired_time: 1.7827384472, propagate_forward_time: 42.1001586914, propagate_backward_time: 61.6061363220, weight_update_time: 0.3375216126, other_time: 5.2088890076
epoch time: 9.5380544662s, input_fired_time: 0.2415395230, output_fired_time: 0.0000000000, propagate_forward_time: 8.5190076828, propagate_backward_time: 0.0000000000, weight_update_time: 0.0000000000, other_time: 0.7775068283
validation[          , genome  6290] predictions:    9764/  10000 (97.64%), best:    9797/10000 (97.97%), error:       817.33514, best error:       697.63519 on epoch:    74, epoch:   89/100, mu: 0.9848997593, learning_rate: 0.0001301099, weight_decay: 0.0000052044

epoch time: 112.9174575806s, input_fired_time: 1.8994307518, output_fired_time: 1.7803362608, propagate_forward_time: 42.2582740784, propagate_backward_time: 61.8315162659, weight_update_time: 0.3345955908, other_time: 5.1478996277
epoch time: 9.4330148697s, input_fired_time: 0.2649020255, output_fired_time: 0.0000000000, propagate_forward_time: 8.3515033722, propagate_backward_time: 0.0000000000, weight_update_time: 0.0000000000, other_time: 0.8166093826
validation[          , genome  6290] predictions:    9734/  10000 (97.34%), best:    9797/10000 (97.97%), error:       926.14313, best error:       697.63519 on epoch:    74, epoch:   90/100, mu: 0.9851547480, learning_rate: 0.0001236044, weight_decay: 0.0000049442

epoch time: 111.6679916382s, input_fired_time: 1.8896236420, output_fired_time: 1.7694379091, propagate_forward_time: 41.8611984253, propagate_backward_time: 61.0371971130, weight_update_time: 0.3312703073, other_time: 5.1105308533
epoch time: 9.2638902664s, input_fired_time: 0.2507804334, output_fired_time: 0.0000000000, propagate_forward_time: 8.2273206711, propagate_backward_time: 0.0000000000, weight_update_time: 0.0000000000, other_time: 0.7857894897
validation[          , genome  6290] predictions:    9781/  10000 (97.81%), best:    9797/10000 (97.97%), error:       752.60931, best error:       697.63519 on epoch:    74, epoch:   91/100, mu: 0.9853969812, learning_rate: 0.0001174242, weight_decay: 0.0000046970

epoch time: 111.5701751709s, input_fired_time: 1.9109945297, output_fired_time: 1.7794283628, propagate_forward_time: 41.6893043518, propagate_backward_time: 61.0828361511, weight_update_time: 0.3394234180, other_time: 5.1076087952
epoch time: 9.4370851517s, input_fired_time: 0.2748768032, output_fired_time: 0.0000000000, propagate_forward_time: 8.3273315430, propagate_backward_time: 0.0000000000, weight_update_time: 0.0000000000, other_time: 0.8348770142
validation[          , genome  6290] predictions:    9748/  10000 (97.48%), best:    9797/10000 (97.97%), error:       873.83240, best error:       697.63519 on epoch:    74, epoch:   92/100, mu: 0.9856271148, learning_rate: 0.0001115530, weight_decay: 0.0000044621

epoch time: 111.1181259155s, input_fired_time: 1.9274129868, output_fired_time: 1.7590010166, propagate_forward_time: 41.5886268616, propagate_backward_time: 60.6562767029, weight_update_time: 0.3366369009, other_time: 5.1868019104
epoch time: 9.3264579773s, input_fired_time: 0.2514249682, output_fired_time: 0.0000000000, propagate_forward_time: 8.3104658127, propagate_backward_time: 0.0000000000, weight_update_time: 0.0000000000, other_time: 0.7645673752
validation[          , genome  6290] predictions:    9777/  10000 (97.77%), best:    9797/10000 (97.97%), error:       749.56769, best error:       697.63519 on epoch:    74, epoch:   93/100, mu: 0.9858457446, learning_rate: 0.0001059753, weight_decay: 0.0000042390

epoch time: 111.6294326782s, input_fired_time: 1.8775595427, output_fired_time: 1.7441399097, propagate_forward_time: 41.9135322571, propagate_backward_time: 61.0803756714, weight_update_time: 0.3216973543, other_time: 5.0138320923
epoch time: 9.1381263733s, input_fired_time: 0.2340359539, output_fired_time: 0.0000000000, propagate_forward_time: 8.1665878296, propagate_backward_time: 0.0000000000, weight_update_time: 0.0000000000, other_time: 0.7375030518
validation[          , genome  6290] predictions:    9796/  10000 (97.96%), best:    9797/10000 (97.97%), error:       744.96313, best error:       697.63519 on epoch:    74, epoch:   94/100, mu: 0.9860534668, learning_rate: 0.0001006766, weight_decay: 0.0000040271

epoch time: 112.9492340088s, input_fired_time: 1.9321575165, output_fired_time: 1.7691963911, propagate_forward_time: 42.3490676880, propagate_backward_time: 61.6536178589, weight_update_time: 0.3419484198, other_time: 5.2451934814
epoch time: 9.2631664276s, input_fired_time: 0.2402825654, output_fired_time: 0.0000000000, propagate_forward_time: 8.2535352707, propagate_backward_time: 0.0000000000, weight_update_time: 0.0000000000, other_time: 0.7693481445
validation[          , genome  6290] predictions:    9770/  10000 (97.70%), best:    9797/10000 (97.97%), error:       801.05853, best error:       697.63519 on epoch:    74, epoch:   95/100, mu: 0.9862508178, learning_rate: 0.0000956427, weight_decay: 0.0000038257

epoch time: 111.1019744873s, input_fired_time: 1.8902328014, output_fired_time: 1.7241551876, propagate_forward_time: 41.5829811096, propagate_backward_time: 60.7876815796, weight_update_time: 0.3280669749, other_time: 5.1169204712
epoch time: 9.4230937958s, input_fired_time: 0.2539342046, output_fired_time: 0.0000000000, propagate_forward_time: 8.3628807068, propagate_backward_time: 0.0000000000, weight_update_time: 0.0000000000, other_time: 0.8062791824
validation[          , genome  6290] predictions:    9780/  10000 (97.80%), best:    9797/10000 (97.97%), error:       790.48145, best error:       697.63519 on epoch:    74, epoch:   96/100, mu: 0.9864382744, learning_rate: 0.0000908606, weight_decay: 0.0000036344

epoch time: 110.7881927490s, input_fired_time: 1.8333928585, output_fired_time: 1.7085223198, propagate_forward_time: 41.4646759033, propagate_backward_time: 60.7756881714, weight_update_time: 0.3260186315, other_time: 5.0059204102
epoch time: 9.5126132965s, input_fired_time: 0.2562933564, output_fired_time: 0.0000000000, propagate_forward_time: 8.4619665146, propagate_backward_time: 0.0000000000, weight_update_time: 0.0000000000, other_time: 0.7943534851
validation[          , genome  6290] predictions:    9765/  10000 (97.65%), best:    9797/10000 (97.97%), error:       797.72809, best error:       697.63519 on epoch:    74, epoch:   97/100, mu: 0.9866163731, learning_rate: 0.0000863176, weight_decay: 0.0000034527

epoch time: 110.8965454102s, input_fired_time: 1.8481662273, output_fired_time: 1.6942945719, propagate_forward_time: 41.6598854065, propagate_backward_time: 60.7392654419, weight_update_time: 0.3225522339, other_time: 4.9549331665
epoch time: 9.5880832672s, input_fired_time: 0.2554113269, output_fired_time: 0.0000000000, propagate_forward_time: 8.5118484497, propagate_backward_time: 0.0000000000, weight_update_time: 0.0000000000, other_time: 0.8208236694
validation[          , genome  6290] predictions:    9795/  10000 (97.95%), best:    9797/10000 (97.97%), error:       735.86542, best error:       697.63519 on epoch:    74, epoch:   98/100, mu: 0.9867855310, learning_rate: 0.0000820017, weight_decay: 0.0000032801

epoch time: 112.4573745728s, input_fired_time: 1.9184457064, output_fired_time: 1.7665814161, propagate_forward_time: 42.0647087097, propagate_backward_time: 61.4660377502, weight_update_time: 0.3337839842, other_time: 5.2416038513
epoch time: 9.3724412918s, input_fired_time: 0.2450537384, output_fired_time: 0.0000000000, propagate_forward_time: 8.3330917358, propagate_backward_time: 0.0000000000, weight_update_time: 0.0000000000, other_time: 0.7942962646
validation[          , genome  6290] predictions:    9711/  10000 (97.11%), best:    9797/10000 (97.97%), error:      1003.89801, best error:       697.63519 on epoch:    74, epoch:   99/100, mu: 0.9869462252, learning_rate: 0.0000779016, weight_decay: 0.0000031161

epoch time: 111.1183319092s, input_fired_time: 1.8214440346, output_fired_time: 1.6662919521, propagate_forward_time: 41.7828254700, propagate_backward_time: 60.9072456360, weight_update_time: 0.3144322932, other_time: 4.9405326843
epoch time: 9.2704591751s, input_fired_time: 0.2340340763, output_fired_time: 0.0000000000, propagate_forward_time: 8.2855224609, propagate_backward_time: 0.0000000000, weight_update_time: 0.0000000000, other_time: 0.7509021759
validation[          , genome  6290] predictions:    9776/  10000 (97.76%), best:    9797/10000 (97.97%), error:       757.72302, best error:       697.63519 on epoch:    74, epoch:  100/100, mu: 0.9870989323, learning_rate: 0.0000740065, weight_decay: 0.0000029603

evaluating best weights on full training data.
evaluting training set with running mean/variance:
epoch time: 47.1707153320s, input_fired_time: 1.2212773561, output_fired_time: 0.0000000000, propagate_forward_time: 41.7581863403, propagate_backward_time: 0.0000000000, weight_update_time: 0.0000000000, other_time: 4.1912498474
best training[          , genome  6290] predictions:   49710/  50000 (99.42%), best:    9797/10000 (97.97%), error:      1143.61755, best error:       697.63519 on epoch:    74, epoch:  101/100, mu: 0.9872440100, learning_rate: 0.0000703062, weight_decay: 0.0000028122
evaluating best weights on test data.
evaluting test set with running mean/variance:
epoch time: 9.5084514618s, input_fired_time: 0.2519589961, output_fired_time: 0.0000000000, propagate_forward_time: 8.3699522018, propagate_backward_time: 0.0000000000, weight_update_time: 0.0000000000, other_time: 0.8865404129
   testing[          , genome  6290] predictions:    9816/  10000 (98.16%), best:    9797/10000 (97.97%), error:       607.13910, best error:       697.63519 on epoch:    74, epoch:  101/100, mu: 0.9872440100, learning_rate: 0.0000703062, weight_decay: 0.0000028122
backpropagation finished successfully!
00:19:04 (9736): called boinc_finish(0)

</stderr_txt>
]]>