fcampelo
10-13-2009, 11:21 AM
Hello all!
Electrical engineer here, currently doing research in evolutionary optimization.
I'm having some doubts regarding the design of a factorial experiment, and I suppose you guys could maybe shed some light on the subject. Here's a description of what I'm trying to do:
I have an algorithm and am trying to evaluate the main effects and interactions of its operators (lets call them A, B and C), each having three different instances, over a set of 25 test problems. The problem is that, given the stochastic nature of the evolutionary algorithms, I have to run multiple replicates of each instance of the algorithm over each problem, in order to obtain average performance measures. In short, I have:
factor A: 3 levels
factor B: 3 levels
factor C: 3 levels
nprobs = 25 problems
nruns = 30 runs/problem/algorithm instance
I was thinking of using the mean (or median) performance value of each instance over each problem as the data points for a factorial design. In this scenario I would have a 3^3 factorial experiment, with 25 (number of problems) replicates on each cell of the ANOVA (or Kruskal-Wallis, if the errors are not i.i.d. ~N(0,sigma^2) ); each of these data points would be representing the average performance over a given problem.
My question is: can I do this? Can estimated mean values (averaged over a large number of samples) be used as single data points in an ANOVA design? How does it affect the validity of the results? :confused:
Thanks in advance,
Felipe
Electrical engineer here, currently doing research in evolutionary optimization.
I'm having some doubts regarding the design of a factorial experiment, and I suppose you guys could maybe shed some light on the subject. Here's a description of what I'm trying to do:
I have an algorithm and am trying to evaluate the main effects and interactions of its operators (lets call them A, B and C), each having three different instances, over a set of 25 test problems. The problem is that, given the stochastic nature of the evolutionary algorithms, I have to run multiple replicates of each instance of the algorithm over each problem, in order to obtain average performance measures. In short, I have:
factor A: 3 levels
factor B: 3 levels
factor C: 3 levels
nprobs = 25 problems
nruns = 30 runs/problem/algorithm instance
I was thinking of using the mean (or median) performance value of each instance over each problem as the data points for a factorial design. In this scenario I would have a 3^3 factorial experiment, with 25 (number of problems) replicates on each cell of the ANOVA (or Kruskal-Wallis, if the errors are not i.i.d. ~N(0,sigma^2) ); each of these data points would be representing the average performance over a given problem.
My question is: can I do this? Can estimated mean values (averaged over a large number of samples) be used as single data points in an ANOVA design? How does it affect the validity of the results? :confused:
Thanks in advance,
Felipe