arno klein's Mindboggle

back to Papers

Evaluation of 14 nonlinear deformation algorithms applied to human brain MRI registration.

Arno Klein, Jesper Andersson, Babak A. Ardekani, John Ashburner, Brian Avants, Ming-Chang Chiang,
Gary E. Christensen, D. Louis Collins, James Gee, Pierre Hellier, Joo Hyun Song, Mark Jenkinson, Claude Lepage,
Daniel Rueckert, Paul Thompson, Tom Vercauteren, Roger P. Woods, J. John Mann, Ramin V. Parsey.

2009. NeuroImage. 46(3): 786-802.

[PubMedCentral] [PDF] [doi:10.1016/j.neuroimage.2008.12.037]

Supplementary Material (12.4 MB PDF)

Corrigenda (1): Table 2 header should read "Volume (voxels)" instead of "Volume (mm)."
Note: In this article, registration methods are ranked with respect to the top-ranking method, not with each other. Be careful not to conclude that one method performed more accurately than another if their rank test means are within a standard deviation of one another.

Code (Matlab and Python):

Compare (and plot) evaluations of the registration methods: evaluate_compare.m
Warp evaluations: evaluate_warps_main.m, evaluate_warps.m
Distance error evaluations: evaluate_distances_main.m, evaluate_distances.m
Label boundaries (for distance errors): make_boundaries_main.m, make_boundaries.m
"Paired indifference-zone" ranking: rank_indifferencezone.m *Added (2010/07/06) standalone python algorithm: rank_paired_indifferencezone.py
Permutation tests for ranking: permutation_test.m
ANTS shell script (created after the study): ants.sh
IRTK parameter file: IRTK_parameters_2.5mm.txt
FNIRT parameter file: FNIRT_schedule.cnf
ANIMAL parameter file: ANIMAL_parameters.pl
Matlab code for composing SPM Normalize and DARTEL transforms: SPM_code.tar.gz
Call preprocessing scripts and registration software packages (Python): run_tests.txt

Data:

CUMC12 data, IBSR18 data, and MGH10 data and information
Rigidly registered pairs of LPBA40 images (3.1 GB) and label volumes (263 MB) and label list
Freesurfer-derived LPBA40 surfaces (6.3 MB)

Results:

Tables of UNION overlap values (intersection of automated and manual labels over their union)

for each region by registration method, averaged across all brain pairs per label set:
Column-sortable HTML: LPBA40, IBSR18, CUMC12, MGH10

Tables of TARGET overlap values (intersection of automated and manual labels over manual labels)

for each region by registration method, averaged across all brain pairs per label set:
Comma-delimited (.csv): LPBA40, IBSR18, CUMC12, MGH10
Column-sortable HTML: LPBA40, IBSR18, CUMC12, MGH10

Example: LPBA40 set, lighter boxes denote higher accuracy (click on column headers to sort):

Region	FLIRT	AIR	ANIMAL	ART	D.Demons	FNIRT	IRTK	JRD-fluid	ROMEO	SICLE	SyN	SPM_N*	SPM_N	SPM_US	SPM_D
L sup. frontal gyrus	74	78	73	82	80	81	80	83	81	71	81	79	76	79	78
R sup. frontal gyrus	74	78	75	81	80	80	79	82	79	70	81	78	76	78	76
L middle frontal gyrus	71	75	75	79	77	79	79	78	78	71	79	77	76	77	76
R middle frontal gyrus	71	74	75	78	77	74	78	78	77	72	78	77	76	77	74
L inf. frontal gyrus	64	71	73	73	72	72	73	74	72	65	74	72	73	72	70
R inf. frontal gyrus	65	69	69	73	72	70	73	74	72	67	73	72	72	72	69
L precentral gyrus	60	63	71	73	68	71	73	69	67	55	74	68	51	70	72
R precentral gyrus	60	62	69	72	69	70	72	70	68	58	73	66	53	70	71
L middle orbitofrontal gyrus	56	66	65	69	67	68	68	69	67	63	69	66	66	66	64
R middle orbitofrontal gyrus	58	66	67	70	67	64	68	69	67	64	69	66	66	66	64
L lateral orbitofrontal gyrus	47	57	61	61	58	59	60	63	59	52	61	59	59	57	57
R lateral orbitofrontal gyrus	46	55	57	59	55	55	57	60	58	51	59	56	56	55	54
L gyrus rectus	56	65	58	70	67	70	64	68	66	56	69	64	61	65	62
R gyrus rectus	59	67	61	71	68	73	65	70	68	58	71	66	64	66	64
L postcentral gyrus	51	56	63	67	61	66	66	62	60	50	68	61	45	65	66
R postcentral gyrus	54	56	61	67	63	66	66	65	62	54	68	61	47	65	68
L sup. parietal gyrus	65	66	70	72	71	71	71	73	71	62	73	69	60	70	69
R sup. parietal gyrus	65	66	71	72	71	70	71	73	71	64	73	70	62	71	69
L supramarginal gyrus	54	60	65	66	64	66	65	65	63	48	68	64	60	66	65
R supramarginal gyrus	54	59	59	65	63	65	65	64	63	52	67	64	61	65	63
L angular gyrus	56	58	60	64	63	63	62	65	62	49	64	62	58	63	62
R angular gyrus	56	61	59	66	65	64	65	66	65	52	66	64	62	65	63
L precuneus	62	63	60	67	66	67	66	67	65	58	67	63	58	66	63
R precuneus	63	65	64	69	67	69	67	69	66	59	69	65	60	67	64
L sup. occipital gyrus	47	54	56	62	58	59	59	60	58	44	61	57	49	59	58
R sup. occipital gyrus	46	52	55	59	55	57	57	58	56	45	59	55	45	56	56
L middle occipital gyrus	58	63	70	69	69	70	66	70	68	57	69	68	57	68	65
R middle occipital gyrus	59	64	70	69	68	70	66	70	69	60	68	67	56	67	65
L inf. occipital gyrus	52	60	72	69	67	68	63	70	67	61	68	67	41	65	63
R inf. occipital gyrus	53	63	72	69	66	70	63	71	68	63	68	66	40	66	64
L cuneus	52	62	56	67	63	69	63	65	65	55	67	64	56	64	64
R cuneus	53	61	59	65	62	67	63	64	62	54	66	63	54	63	62
L sup. temporal gyrus	60	70	72	76	72	74	77	75	71	66	77	71	70	71	71
R sup. temporal gyrus	61	72	72	77	73	74	77	76	73	68	78	72	71	72	72
L middle temporal gyrus	55	62	64	68	66	68	66	67	64	62	68	65	60	66	64
R middle temporal gyrus	57	64	65	70	68	70	69	70	68	66	71	67	64	68	66
L inf. temporal gyrus	53	60	67	68	66	70	66	68	65	60	67	64	52	64	62
R inf. temporal gyrus	56	61	67	68	66	71	67	70	67	63	68	65	56	65	64
L parahippocampal gyrus	54	66	65	73	68	69	72	69	67	56	72	64	35	68	65
R parahippocampal gyrus	55	65	64	71	67	68	72	68	65	55	71	64	35	67	64
L lingual gyrus	60	66	67	72	70	70	70	70	69	66	73	67	41	68	69
R lingual gyrus	63	69	71	74	71	72	73	72	70	67	75	69	41	70	70
L fusiform gyrus	55	66	68	73	69	71	69	68	67	62	71	66	45	68	66
R fusiform gyrus	58	66	68	73	68	72	69	70	67	62	71	67	45	68	65
L insular cortex	64	73	70	81	76	73	81	77	74	63	79	70	69	76	75
R insular cortex	61	71	66	80	74	73	80	74	73	63	78	69	69	74	72
L cingulate gyrus	62	67	60	73	68	69	71	69	68	58	72	62	64	67	66
R cingulate gyrus	61	66	60	72	68	70	70	68	66	56	71	62	64	67	66
L caudate	60	66	60	78	73	68	78	71	73	58	75	69	68	69	69
R caudate	60	68	58	78	73	70	77	71	72	58	75	68	70	70	72
L putamen	64	73	64	81	76	77	78	71	73	67	78	67	67	77	72
R putamen	64	73	64	80	76	75	77	71	73	69	78	69	68	77	71
L hippocampus	59	70	71	77	73	75	75	71	71	61	75	69	40	74	70
R hippocampus	62	70	72	77	73	75	75	72	71	61	75	68	48	73	70
cerebellum	81	66	85	88	85	86	82	83	86	82	88	84	28	86	80
brainstem	78	69	79	81	81	82	78	79	81	77	80	77	37	79	76

Registration evaluation studies conducted since this article was published:

EMPIRE: Evaluation of Methods for Pulmonary Image Registration 2010 (MICCAI 2010 challenge workshop)