Results for the Phase0Testing simulations

As of Nov 27, 2008, fourteen people replied to the Phase0Testing simulations. In the following the results for the blind run on the noise.cat catalogue are presented in two ways:

• plots showing the model redshift vs. the estimated redshift
• Mean and RMS values of the quantity Δz=z_model-z_phot together with outlier rates. We calculated the mean and the RMS of the redshift differences twice, once with an iterative 5-sigma outliers rejection and once rejecting all objects with Δz>0.1.

No cuts were applied to the various quality indicators put out by the different codes.

First, see as a reference the output of the code "Le Phare" when Stephane ran it with the noise.cat catalogue.

Arnouts, Stephane (Le Phare, template-based):

• z_model vs. z_phot:
  

• statistics:
  • iterative 5-σ outlier rejection: 0.423916% outliers, Δz=1.77827e-05 ⁺/_- 0.00961287
  • Δz>0.1 outlier rejection: 0.0441579% outliers, Δz=3.00583e-05 ⁺/_- 0.0103438

In the following, the results from all other people are presented. Note, that suboptimal results do not indicate necessarily an inaccurate code in general. Deviations should only be interpreted in the way, that this particular code does not agree perfectly well with Stephane's "Le Phare" code, which was used for the creation of the simulations.

Abdalla, Filipe (ANNz, neural-network code, see Collister & Lahav 2004):

• z_model vs. z_phot:
  
• statistics:
  • iterative 5-σ outlier rejection: 0.22079% outliers, Δz=9.15565e-05 ⁺/_- 0.0106914
  • Δz>0.1 outlier rejection: 0.0176632% outliers, Δz=9.46557e-05 ⁺/_- 0.0110893

Assef, Roberto (LRT, template-based code, optimised for low-resolution templates, see Assef, et al. 2008):

• z_model vs. z_phot:
  
• statistics:
  • iterative 5-σ outlier rejection: 0.750684% outliers, Δz=0.000129027 ⁺/_- 0.0121307
  • Δz>0.1 outlier rejection: 0.247284% outliers, Δz=0.000131031 ⁺/_- 0.0134385

Brammer, Gabriel (EAZY, template-based code, see Brammer et al. 2008):

• z_model vs. z_phot:
  
• statistics:
  • iterative 5-σ outlier rejection: 0.203126% outliers, Δz=-0.000555487 ⁺/_- 0.0118434
  • Δz>0.1 outlier rejection: 0% outliers, Δz=-0.000628809 ⁺/_- 0.0123139

Carliles, Sam (empirical code, ensemble of "randomized" regression trees trained on bootstrap samples from the training set):

• z_model vs. z_phot:
  
• statistics:
  • iterative 5-σ outlier rejection: 0.256116% outliers, Δz=7.36852e-05 ⁺/_- 0.0125069
  • Δz>0.1 outlier rejection: 0.00883158% outliers, Δz=0.000128865 ⁺/_- 0.013035

Coe, Dan (BPZ, template-based code, see Benitez 2000):

• z_model vs. z_phot:
  
• statistics:
  • iterative 5-σ outlier rejection: 0.291442% outliers, Δz=-0.00492117 ⁺/_- 0.0102579
  • Δz>0.1 outlier rejection: 0.0264947% outliers, Δz=-0.00499912 ⁺/_- 0.0107985

Dahlen, Tomas (template-based code):

• z_model vs. z_phot:
  
• statistics:
  • iterative 5-σ outlier rejection: 0.397421% outliers, Δz=8.77815e-05 ⁺/_- 0.0108293
  • Δz>0.1 outlier rejection: 0.0176632% outliers, Δz=0.000128081 ⁺/_- 0.0116462

Feldmann, Robert (ZEBRA, template-based code, see Feldmann et al. 2006):

• z_model vs. z_phot:
  
• statistics:
  • iterative 5-σ outlier rejection: 0.512232% outliers, Δz=-6.39148e-05 ⁺/_- 0.00970347
  • Δz>0.1 outlier rejection: 0.0618211% outliers, Δz=-0.000118063 ⁺/_- 0.0105582

Gerdes, David (training-set-based approach using boosted decision trees):

• z_model vs. z_phot:
  
• statistics:
  • iterative 5-σ outlier rejection: 0.415084% outliers, Δz=-0.00368305 ⁺/_- 0.0185263
  • Δz>0.1 outlier rejection: 0.38859% outliers, Δz=-0.00365651 ⁺/_- 0.0185952

Gillis, Bryan (ZEBRA, template-based code, see Feldmann et al. 2006):

• z_model vs. z_phot:
  
• statistics:
  • iterative 5-σ outlier rejection: 0.468074% outliers, Δz=-0.00481047 ⁺/_- 0.0100153
  • Δz>0.1 outlier rejection: 0.0441579% outliers, Δz=-0.00487675 ⁺/_- 0.0108845
• Notes on setting up ZEBRA for running this data: ZebraNotes

Kotulla, Ralf (GALEV, template-based code, see Kotulla et al. 2009, IRAC bands not included):

• z_model vs. z_phot:
  
• statistics:
  • iterative 5-σ outlier rejection: 0.344432% outliers, Δz=-8.78235e-05 ⁺/_- 0.0132298
  • Δz>0.1 outlier rejection: 0.0529895% outliers, Δz=-6.14209e-05 ⁺/_- 0.0139058

Li, Tornado (empirical code, polynomial fitting):

• z_model vs. z_phot:
  
• statistics:
  • iterative 5-σ outlier rejection: 1.75748% outliers, Δz=0.000672984 ⁺/_- 0.0191082
  • Δz>0.1 outlier rejection: 1.66917% outliers, Δz=0.000654796 ⁺/_- 0.0193264

Miralles, Joan-Marc (Hyperz, template-based code, see Bolzonella et al. 2000):

• z_model vs. z_phot:
  
• statistics:
  • iterative 5-σ outlier rejection: 0.803674% outliers, Δz=-0.00185141 ⁺/_- 0.0120707
  • Δz>0.1 outlier rejection: 0.185463% outliers, Δz=-0.00181295 ⁺/_- 0.0134263

Purger, Norbert (empirical neural-network & template-based code):

• z_model vs. z_phot (empirical):
  
• statistics (empirical):
  • iterative 5-σ outlier rejection: 0.141305% outliers, Δz=0.000155656 ⁺/_- 0.0169082
  • Δz>0.1 outlier rejection: 0.0529895% outliers, Δz=0.000190863 ⁺/_- 0.0171598

• z_model vs. z_phot (template-based):
  
• statistics (template-based):
  • iterative 5-σ outlier rejection: 0.3356% outliers, Δz=-0.00467166 ⁺/_- 0.0102862
  • Δz>0.1 outlier rejection: 0.0529895% outliers, Δz=-0.00466801 ⁺/_- 0.0109

Singal, Jack (neural-network code):

• z_model vs. z_phot:
  
• statistics:
  • iterative 5-σ outlier rejection: 0% outliers, Δz=0.0114871 ⁺/_- 0.0753925
  • Δz>0.1 outlier rejection: 18.2019% outliers, Δz=-0.00467356 ⁺/_- 0.0494039

-- HendrikHildebrandt - 29 June, 2009