Effex app documentation – Filtering controls

Docs: Define your experiment

Mon, 01 Jan 0001 00:00:00 +0000

In this tab, you specify the number of factors in your experiment, the type of these factors and the target model that you want to fit.

Experimental factors

Enter the number of three-level factors and the number of two-level factors involved in your experiment using the input boxes.

You can include two different kinds of experimental factors: two-level categorical or continuous factors, and three-level continuous factors. The experimental factors are also called independent variables, or controllable factors. For each type mentioned, it is important to know when to use them to model the effects of the experimental factors:

Two-level factors are used to model experimental factors that can take two different values.
- Two-level factors can be used with both continuous and categorical factors. One the one hand, when the factor is quantitative, the first level corresponds to the lower level and the second level to the upper level of the interval wherein the factor varies.
- Examples: a categorical factor that indicates if the experiment is performed by either machine A or machine B, or a continuous factor that indicates the low and high value of temperature investigated at two levels only.
- Technical considerations (click to unfold)
  - The two different values of two-level factors are coded as $-1$ and $1$.
  - It is not possible to estimate the quadratic effect of a continuous experimental variable modeled as a two-level factor.
  - For calculation of prediction variances, two-level factors are taken as categorical.
Three-level factors are used to model quantitative experimental factors.
- The factors are assumed to vary within a closed continuous interval and three values are considered when planning the experimental design: design: the lowest and highest value of the interval and the value in between these values.
- Example: a factor that indicates the temperature of an experiment which can take values within a given range.
- Technical considerations (click to unfold)
  - The three values are coded as:
    
    Low value: $-1$
    
    Medium value: $0$
    
    High value: $1$
  - Three-level factors allow the user to study quadratic effects in addition to the main effects and two-factor interaction effects.

The software database contains hundreds of thousands of experimental designs that contain only three-level factors. The most important family of experimental designs are OMARS designs ¹. For these designs, all main effects can be independently estimated from each other and from any other second-order effect (2-factor interaction effects and quadratic effects). Standard designs such as Definitive Screening Designs, Central Composite Designs and Box-Behnken designs are examples of OMARS designs.

When you combine two- and three-level factors, you will end up with what is known as a mixed level design. To the best of our knowledge, our database is the only one with OMARS mixed level designs². For these designs, all the three level and two level factors have all main effects orthogonal to each other and to all second order effects.

Blocking

Very often there are blocking factors that can potentially influence the response. Check this box if you would like to use an additional blocking factor with your design.

Blocking is very common in experiments and is often implemented when groups of experimental runs are performed under different or non-homogenous experimental conditions. For example, a number of experiments could be run during the course of multiple days. In such a case, the ‘day’ in which the runs are performed could be treated as a blocking factor. This can be done by introducing a blocking factor in the design. When the blocking option is checked, the software always produces designs where the blocking factor is orthogonal to all main effects of the original factors and hence the final design selected will allow the user to study the main effects from the original factors independent from the blocking factor.

The controls to include a blocking factor in an experimental designs are:

Number of blocks and block size.
- Most often the number of runs that can be performed in a single block (group) of experimentation is known in advance. For example, the blocking factor can be a day and the block size is determined by how many tests can be carried out in one day.
- Once the block size is fixed, you can use the range slider to set the number of blocks. This will allow you to compare experimental plans with different number of blocks with each other, and assess what the trade-off between the run size and the quality of the design is.
- The number of runs of the experimental design equals the block size multiplied by the number of blocks.
- Example: in a cake baking experiment, if it can be expected that different baking ovens used will produce non-homogenous baking conditions, then in such a case, the oven used can be treated as a blocking variable. In such a case, the number of blocks would be set as the number of different ovens used during the experiments.
Intrablock correlation coefficient is used to indicate how different the blocks are from each other. It equals the ratio between the variance between the blocks and the error variance.
- A value of EXTREME corresponds to the situation when the blocks are expected to be very different from each other and they will be modeled as a fixed block effect.
- The values LOW, MEDIUM and HIGH correspond to a coefficient equal to 0.1, 1 and 10, respectively. Any of these three options indicate that the block effect will be treated as a random block effect.
- The default value in the software is MEDIUM.
- Technical considerations (click to unfold)
  - To define the nature of the block, we need to consider two sources of variation. On the one hand, two observations coming from runs within the same block are expected to differ from each other according to the residual error variance. On the other hand, two observations coming from different blocks are, additionally, expected to differ from each other according to the sum of the residual error variance and the variance between blocks. The ratio between the variance between blocks and the error variance is known as the intrablock correlation coefficient.
  - As mentioned earlier, the blocking factor can be treated as a fixed effect or a random effect. It is sensible to set the blocking factor as a fixed effect if the groups of runs in each individual block is expected to produce responses that are drastically different than the runs in another block. Modeling the blocking factor as a fixed effect, uses more degrees of freedom for estimation since each group or block is treated as a separate effect. However, if the runs in different blocks are considered to be only marginally different, then it is recommended to treat the blocking factor as a random effect which frees up a few degrees of freedom to estimate other effects involving the original factors which are often more important.

Strategy: model of interest and efficiencies

This important section of the controls allows the user to set the target statistical model, which indicates how many terms the linear model will consist of. After setting the model of interest with the slider, the user will be able to set a minimal value for both the D- and A-efficiency.

Below you can find more detailed information on how these controls work and the statistical theory behind them.

The slider has three different fixed positions:
- ME model: this includes all the main effects of the original factors
- ME + IE model: this includes all the main effects and two-factor interactions
- ME + SOE model: this includes all the main effects and second order effects (two-factor interactions and quadratic effects).
- Note that the values are nested, which is indicated by the coloring of the slider bar which starts from the left side. For example, when a ME+IE model is selected, the slider is colored including also a ME model, which indicates that a ME model is also estimable.
- Technical considerations (click to unfold)
  
  Consider an experiment with $m=m_1+m_2$ factors, with $m_1$ 3-level factors and $m_2$ 2-level factors. The designs in our catalog allow the estimation of a linear model that relates the factors and a response. The choice of the model needs to be done before selecting a design, as, for example, not all designs allow fitting all second-order effects.
  
  A ME model is given by the following equation:
  
  $ y = \beta_0 + \sum_{i=1}^{m} \beta_i x_i + \varepsilon $,
  
  where $y$ is the response, $\beta_0$ corresponds to the intercept, $\beta_i$ corresponds to the main effect of the $i$th factor, $x_i$ is the $i$th factor level, and $\varepsilon$ is the random error which is assumed to follow a normal distribution.
  
  A ME + IE model is given by the following equation:
  
  $ y = \beta_0 + \sum_{i=1}^{m} \beta_i x_i + \sum_{i=1}^{m-1} \sum_{j=i+1}^{m} \beta_{ij} x_i x_j + \varepsilon $,
  
  where $\beta_{ij}$ is the two-factor interaction effect between the $i$th and the $j$th factors.
  
  A ME + SOE model is given by the following equation:
  
  $ y = \beta_0 + \sum_{i=1}^{m} \beta_i x_i + \sum_{i=1}^{m-1} \sum_{j=i+1}^{m} \beta_{ij} x_i x_j + \sum_{i=1}^{m_1} \beta_{ii} x_i^2 + \varepsilon $,
  
  where $\beta_{ii}$ is the quadratic effect of the $i$th factor.
The two sliders below allow the user to set a minimum limit on the D and A efficiencies for the design for the chosen model of interest.
- This model will include the blocking factor if selected.
- The next two sliders are used to set a lower limit on the D and A efficiencies for the model chosen. The slider ranges from 0 to 100%.
- Technical considerations (click to unfold)
  The D and A-efficiencies are calculated as follows:
  
  $\text{D-efficiency} = \frac{\mathbf{{|X’X|}^{1/p}}}{N}$, and
  
  $\textit{A-efficiency} = \frac{p}{N \times tr\mathbf{{(X’X)}^{-1}}}$, where
  
  $\mathbf{X}$ is model matrix, $p$ is the number of effects of the statistical model (which equals the number of columns of the model matrix), and $N$ is the number of runs.
  
  When the design is organized in blocks, the calculations are different.
  - When blocking factor is treated as a fixed effect:
    
    $\text{D-efficiency} = \frac{\mathbf{{|[XZ]'[XZ]|}^{1/p}}}{N}$
    
    $\textit{A-efficiency} = \frac{p}{N \times tr\mathbf{{([XZ]'[XZ])}^{-1}}}$
    
    where $\mathbf{X}$ is the model matrix selected (ME, ME+IE, or ME+SOE) excluding the intercept, $\mathbf{Z}$ is the dummy coded matrix for the blocking factor, $\mathbf{[XZ]}$ is the concatenated full model matrix , $p$ is the total number of effects and $N$ is the number of runs.
  - When blocking factor is treated as a random effect:
    
    $\textit{D-efficiency} = \frac{\mathbf{{|X’V^{-1}X|}^{1/p}}}{N}$
    
    $\textit{A-efficiency} = \frac{p}{N \times tr\mathbf{{(X’V^{-1}X)}^{-1}}}$
    
    where $\mathbf{X}$ is the model matrix selected (ME, ME+IE, or ME+SOE) including the intercept, $\mathbf{V}$ is the variance-covariance matrix of all responses, p is the total number of effects and N is the number of runs. Here, $\mathbf{V}$ is adapted based on the intrablock correlation setting.
  It is important to note that the efficiency values will generally be low for three-level or mixed-level designs. This is because the efficiency is calculated in comparison to a theoretical optimal for a two level design.
  
  For a more thorough understanding of the concept of blocking, refer to Chapter 7 and 8 of Optimal design of experiments by Peter Goos, Bradley Jones [^gj].

José Núñez Ares & Peter Goos (2020) Enumeration and Multicriteria Selection of Orthogonal Minimally Aliased Response Surface Designs, Technometrics, 62:1, 21-36. ↩︎
José Núñez Ares, Eric Schoen, and Peter Goos. Orthogonal Minimally Aliased Response Surface Designs for Three-Level Quantitative Factors and Two-Level Categorical Factors. Statistica Sinica.-Taiwan 33.1 (2023): 107-126. ↩︎

Docs: Design size, powers and aliasing

Thu, 05 Jan 2017 00:00:00 +0000

Run size

Use the following slider to set a range for minimum and maximum run size for your design of interest. A good advice is to add a couple of runs above and below your target run size to explore the full potential of our database

Power

Use the following slider to specify the minimum power to detect one main effect, one interaction effect or one quadratic effect.
- The base model contains the intercept and all main effects.
- The powers are calculated using a signal-to-noise ratio (SNR) of 1 and a significance level $\alpha=0.05$.
- It is possible to check the powers for different SNR values and thresholds of significance at a later stage for the final chosen design.
- Technical considerations (click to unfold)
  The power to detect an effect is primarily influenced by three things; the signal-to-noise ratio, reliability of the sample size and significance threshold. We will explain all three in this text.
  
  The null hypothesis ($H_o$) is that a certain effect $\beta_i = 0$ and that the following term
  
  $F_o = \frac{\beta_{i}^{2}}{\hat{\sigma}^2\mathbf{(X’X)^{-1}_{ii}}}$
  
  follows a central F-distribution with $1$ and $n-p$ degrees of freedom where $n$ is the number of runs and $p$ is the number of parameters in the model matrix $\mathbf{X}$ including the intercept. In the expression, $\hat{\sigma}^2$ is the estimate of the mean square error from the model and $\hat{\beta}_i$ is the estimate of the size of an effect $i$.
  
  Since we are calculating power for an active effect ($\beta_i \neq 0$), in this case, the true $F$ has a non-central F-distribution, where the non-centrality parameter is calculated as follows:
  
  $\lambda = \frac{(\beta_{i}^{*})^{2}}{\sigma^{2}\mathbf{({X}‘X)^{-1} _{ii}}} = \frac{(SNR)^{2}}{\mathbf{({X}‘X)^{-1} _{ii}}}$ .
  
  Here, $\beta_{i}^{*}$ is the true size of the effect i and $\sigma^2$ is the true value of the standard error. The ratio of the true effect size and true standard error is often referred to as signal-to noise ratio (SNR). For design characterization, the default value for SNR is set to 1.
  
  Next, a significance threshold ($\alpha$) needs to be set. This is the p-value at which a significance conclusion will be drawn. For design characterization, $\alpha$ is set at 0.05.
  
  Summarized steps to calculate power for each effect:
  1. The preset value of SNR (=1) and $\mathbf{(X’X)^{-1}_{ii}}$ is used to calculate the non-centrality parameter $\lambda$.
  2. Calculate ‘Critical F-value’ from a central distribution F-table with $\alpha=0.05$, with $1$ and $n - p$ degrees of freedom.
  3. Obtain the type II error using non central F-distribution(Critical F-value, $1$ , $n - p$, $\lambda)$. The type II error is the probability of accepting the null hypothesis, when the alternative hypothesis is true. The type II error is denoted as $\beta$ (not to be mistaken with the effect size in a statistical model).
    
    $\textit{Power} = 1 - \beta$
  Note: When random block effects are included, the term $\mathbf{(X’V^{-1}X)^{-1}_{ii}}$ is used in step 1.
  
  To determine the power for:
  - Single main effect: the model matrix $\mathbf{X}$ consists of the intercept and all the main effects columns. The minimum power attained for any of the main effect is recorded.
  - Single interaction effect: the model matrix consists of the intercept and all main effects columns and the one interaction effect column. For all designs, all individual interaction effects are tested separately in a base model that includes all main effects and the minimum power attained for any of the interaction effect is recorded. The user can use the slider to set the minimum power requirements to detect a single interaction effect in a base model with all main effects.
  - Single quadratic effect: the model matrix consists of the intercept and all main effects columns and the one quadratic effect column. For all designs, all individual quadratic effects are tested separately in a base model that includes all main effects and the minimum power attained for any of the quadratic effect is recorded. The user can use the slider to set the minimum power requirements to detect a single quadratic effect in a base model with all main effects.
  When fixed block effects are considered, the model matrix will also include dummy coded block effect columns.

Aliasing

Use the following slider to fix the maximum absolute value of the aliasing/correlation between any two second-order effect. The slider ranges from 0 to 1.

Docs: Projection properties

Thu, 05 Jan 2017 00:00:00 +0000

Often, only a limited number of the factors in an experiment turn out to have significant effects. Ideally, the design allows you to obtain a good quality second-order model involving these factors. Such a model includes all their main effects, interaction effects and, in case of 3-level factors, quadratic effects.

Before doing the experiment, you don’t know exactly which of the factors will have significant effects. However, you may have some idea about the maximum number of factors involved.

In general, if you have $p$ factors and believe that at most $q$ of them can show significant effects, there are $\binom{p}{q}$ projections, and a design that performs well on average over all the $q$-factor projections is preferred.
Example: suppose that your design includes 10 factors, and you believe that at most 5 of them can show significant effects. The design has $\binom{10}{5}=252$ different subsets of 5 factors. In other words, there are 252 different projections of the design into 5 factors. A design with good average estimation quality over all 252 projections would serve your turn.

This tab allows you to specify:

The maximum number of factors that you believe to have significant effects
- The controls allow specification for the maximum number in models with only three-level factors, in models with only two-level factors and in models with a combination of three-level and two-level factors.
The minimum values for two criteria to quantify how efficiently the design quantifies a second-order model with this number of factors.
- Both the sliders and the fill-out boxes allow you to specify minimum values for the average D-efficiency, the average A-efficiency, or both.
- Both of these criteria address the statistical uncertainty in the model coefficients. This uncertainty can be expressed as the variance of the individual coefficients and the correlation of pairs of coefficients. The variance of a coefficient measures how precise the coefficient itself is estimated. The correlation among two coefficients measures the amount of cross-talk between them. If two coefficients are highly correlated, you cannot be sure which of the corresponding effects contributes to either coefficient. If they are uncorrelated, the interpretation of the coefficients is unambiguous.
- Technical considerations (click to unfold)
  - A design’s D-efficiency for a particular model addresses the variance of the model coefficients as well as the correlation between these coefficients. Maximizing the D-efficiency simultaneously minimizes the uncertainty in the coefficients themselves and the cross-talk between coefficients.
  - The A-efficiency of a design for a particular model solely addresses the variance of the model coefficients. Maximizing the A-efficiency minimizes the uncertainty in the coefficients, but leaves the amount of cross-talk between coefficients unaddressed.
  - Both kinds of efficiency can attain values between 0% and 100%. An average of 0% means that the model cannot be calculated for any of the projections. An average of 100% means that the models for the projections all have the best possible efficiency for all the projections. If the average is in between, the models don’t all have the best efficiency but neither do they all have the worst possible efficiency.
  - The technical definition of the D-efficiency of a model with p parameters (including intercept) based on a design with N runs is $$D = 100 \cdot \frac{|\mathbf{X}^T\mathbf{X}|^{1/p}}{N}, $$
    where |.| denotes the determinant, and $\mathbf{X}$ is an $N \times p$ model matrix with intercept, and further independent variables corresponding to some or all main effects, interaction effects and quadratic effects. Each effect in the matrix is normalized to a length of $\sqrt{N}$.
  - The technical definition of the A-efficiency of a model with p parameters based on a design with N runs is $$A = 100 \cdot \frac{p}{N \cdot tr[(\mathbf{X}^T\mathbf{X})^{-1}]},$$ where $tr[.]$ denotes the trace of a matrix, $(.)^{-1}$ denotes the inverse of a matrix and $\mathbf{X}$ is defined above.

The two sets of controls allow you to specify projection estimability requirements for two types of subsets of effects at the same time.

Docs: Advanced options

Thu, 05 Jan 2017 00:00:00 +0000

Center points, pure error and uniform precision designs

A center point is a design point where all the three-level factors are set at their middle level. The slider designated ‘Center Points’ allows you to specify a range of numbers of such points.
The slider designated DF for pure error estimation allows you to specify the minimum number of degrees of freedom for a model-free estimate of the random error. This number is determined as the number of runs minus the number of distinct points of the design. If you want to base the statistical tests solely on the model-free estimate, a minimum of 3 DF for pure error estimation is recommended.
- If the design includes only three-level factors, including two or more center points allows for model-free estimation of the random error, also called pure error estimation. The corresponding number of degrees of freedom is one less than the number of center points. If the design includes additional two-level factors, the number of center points is the total over all the combinations of two-level factors. In that case, including more than one center point may or may not allow for pure error estimation.
There are two check boxes to specify conditions on the number of runs at the middle level of the three-level factors. Checking the box Main and quadratic effects ensures that the number of runs at the middle level of any numerical three-level factor is the same. This ensures a roughly equal precision for the main effects and for the quadratic effects in models with just these effects.
Checking the box Interaction effects ensures that, for any pair of numerical three-level factors, the number of runs where one or both of the factors attain their middle level is the same. This ensures an equal precision for the interaction effects in all models with main effects and a single interaction effect.