Effex app documentation – Documentation

Docs: Designs

Thu, 05 Jan 2017 00:00:00 +0000

This table shows the designs you have saved or purchased. A saved design just includes the specifications of a design that you are interested in. These can be deleted. Any purchased design is automatically added to the table. A purchased design cannot be deleted. You can always edit the name and description of a design in My library.

Docs: Summary

Thu, 05 Jan 2017 00:00:00 +0000

Selection of entries in the other tabs. Importantly, this tab informs us about the price of the design.

When you place the pointer of your mouse over the text on the Properties column, you will get an explanation.

The price appears in the last row and it is expressed in credits. The credits are the currency of our software and the way of purchasing designs.

Docs: Define your experiment

Mon, 01 Jan 0001 00:00:00 +0000

In this tab, you specify the number of factors in your experiment, the type of these factors and the target model that you want to fit.

Experimental factors

Enter the number of three-level factors and the number of two-level factors involved in your experiment using the input boxes.

You can include two different kinds of experimental factors: two-level categorical or continuous factors, and three-level continuous factors. The experimental factors are also called independent variables, or controllable factors. For each type mentioned, it is important to know when to use them to model the effects of the experimental factors:

Two-level factors are used to model experimental factors that can take two different values.
- Two-level factors can be used with both continuous and categorical factors. One the one hand, when the factor is quantitative, the first level corresponds to the lower level and the second level to the upper level of the interval wherein the factor varies.
- Examples: a categorical factor that indicates if the experiment is performed by either machine A or machine B, or a continuous factor that indicates the low and high value of temperature investigated at two levels only.
- Technical considerations (click to unfold)
  - The two different values of two-level factors are coded as $-1$ and $1$.
  - It is not possible to estimate the quadratic effect of a continuous experimental variable modeled as a two-level factor.
  - For calculation of prediction variances, two-level factors are taken as categorical.
Three-level factors are used to model quantitative experimental factors.
- The factors are assumed to vary within a closed continuous interval and three values are considered when planning the experimental design: design: the lowest and highest value of the interval and the value in between these values.
- Example: a factor that indicates the temperature of an experiment which can take values within a given range.
- Technical considerations (click to unfold)
  - The three values are coded as:
    
    Low value: $-1$
    
    Medium value: $0$
    
    High value: $1$
  - Three-level factors allow the user to study quadratic effects in addition to the main effects and two-factor interaction effects.

The software database contains hundreds of thousands of experimental designs that contain only three-level factors. The most important family of experimental designs are OMARS designs ¹. For these designs, all main effects can be independently estimated from each other and from any other second-order effect (2-factor interaction effects and quadratic effects). Standard designs such as Definitive Screening Designs, Central Composite Designs and Box-Behnken designs are examples of OMARS designs.

When you combine two- and three-level factors, you will end up with what is known as a mixed level design. To the best of our knowledge, our database is the only one with OMARS mixed level designs². For these designs, all the three level and two level factors have all main effects orthogonal to each other and to all second order effects.

Blocking

Very often there are blocking factors that can potentially influence the response. Check this box if you would like to use an additional blocking factor with your design.

Blocking is very common in experiments and is often implemented when groups of experimental runs are performed under different or non-homogenous experimental conditions. For example, a number of experiments could be run during the course of multiple days. In such a case, the ‘day’ in which the runs are performed could be treated as a blocking factor. This can be done by introducing a blocking factor in the design. When the blocking option is checked, the software always produces designs where the blocking factor is orthogonal to all main effects of the original factors and hence the final design selected will allow the user to study the main effects from the original factors independent from the blocking factor.

The controls to include a blocking factor in an experimental designs are:

Number of blocks and block size.
- Most often the number of runs that can be performed in a single block (group) of experimentation is known in advance. For example, the blocking factor can be a day and the block size is determined by how many tests can be carried out in one day.
- Once the block size is fixed, you can use the range slider to set the number of blocks. This will allow you to compare experimental plans with different number of blocks with each other, and assess what the trade-off between the run size and the quality of the design is.
- The number of runs of the experimental design equals the block size multiplied by the number of blocks.
- Example: in a cake baking experiment, if it can be expected that different baking ovens used will produce non-homogenous baking conditions, then in such a case, the oven used can be treated as a blocking variable. In such a case, the number of blocks would be set as the number of different ovens used during the experiments.
Intrablock correlation coefficient is used to indicate how different the blocks are from each other. It equals the ratio between the variance between the blocks and the error variance.
- A value of EXTREME corresponds to the situation when the blocks are expected to be very different from each other and they will be modeled as a fixed block effect.
- The values LOW, MEDIUM and HIGH correspond to a coefficient equal to 0.1, 1 and 10, respectively. Any of these three options indicate that the block effect will be treated as a random block effect.
- The default value in the software is MEDIUM.
- Technical considerations (click to unfold)
  - To define the nature of the block, we need to consider two sources of variation. On the one hand, two observations coming from runs within the same block are expected to differ from each other according to the residual error variance. On the other hand, two observations coming from different blocks are, additionally, expected to differ from each other according to the sum of the residual error variance and the variance between blocks. The ratio between the variance between blocks and the error variance is known as the intrablock correlation coefficient.
  - As mentioned earlier, the blocking factor can be treated as a fixed effect or a random effect. It is sensible to set the blocking factor as a fixed effect if the groups of runs in each individual block is expected to produce responses that are drastically different than the runs in another block. Modeling the blocking factor as a fixed effect, uses more degrees of freedom for estimation since each group or block is treated as a separate effect. However, if the runs in different blocks are considered to be only marginally different, then it is recommended to treat the blocking factor as a random effect which frees up a few degrees of freedom to estimate other effects involving the original factors which are often more important.

Strategy: model of interest and efficiencies

This important section of the controls allows the user to set the target statistical model, which indicates how many terms the linear model will consist of. After setting the model of interest with the slider, the user will be able to set a minimal value for both the D- and A-efficiency.

Below you can find more detailed information on how these controls work and the statistical theory behind them.

The slider has three different fixed positions:
- ME model: this includes all the main effects of the original factors
- ME + IE model: this includes all the main effects and two-factor interactions
- ME + SOE model: this includes all the main effects and second order effects (two-factor interactions and quadratic effects).
- Note that the values are nested, which is indicated by the coloring of the slider bar which starts from the left side. For example, when a ME+IE model is selected, the slider is colored including also a ME model, which indicates that a ME model is also estimable.
- Technical considerations (click to unfold)
  
  Consider an experiment with $m=m_1+m_2$ factors, with $m_1$ 3-level factors and $m_2$ 2-level factors. The designs in our catalog allow the estimation of a linear model that relates the factors and a response. The choice of the model needs to be done before selecting a design, as, for example, not all designs allow fitting all second-order effects.
  
  A ME model is given by the following equation:
  
  $ y = \beta_0 + \sum_{i=1}^{m} \beta_i x_i + \varepsilon $,
  
  where $y$ is the response, $\beta_0$ corresponds to the intercept, $\beta_i$ corresponds to the main effect of the $i$th factor, $x_i$ is the $i$th factor level, and $\varepsilon$ is the random error which is assumed to follow a normal distribution.
  
  A ME + IE model is given by the following equation:
  
  $ y = \beta_0 + \sum_{i=1}^{m} \beta_i x_i + \sum_{i=1}^{m-1} \sum_{j=i+1}^{m} \beta_{ij} x_i x_j + \varepsilon $,
  
  where $\beta_{ij}$ is the two-factor interaction effect between the $i$th and the $j$th factors.
  
  A ME + SOE model is given by the following equation:
  
  $ y = \beta_0 + \sum_{i=1}^{m} \beta_i x_i + \sum_{i=1}^{m-1} \sum_{j=i+1}^{m} \beta_{ij} x_i x_j + \sum_{i=1}^{m_1} \beta_{ii} x_i^2 + \varepsilon $,
  
  where $\beta_{ii}$ is the quadratic effect of the $i$th factor.
The two sliders below allow the user to set a minimum limit on the D and A efficiencies for the design for the chosen model of interest.
- This model will include the blocking factor if selected.
- The next two sliders are used to set a lower limit on the D and A efficiencies for the model chosen. The slider ranges from 0 to 100%.
- Technical considerations (click to unfold)
  The D and A-efficiencies are calculated as follows:
  
  $\text{D-efficiency} = \frac{\mathbf{{|X’X|}^{1/p}}}{N}$, and
  
  $\textit{A-efficiency} = \frac{p}{N \times tr\mathbf{{(X’X)}^{-1}}}$, where
  
  $\mathbf{X}$ is model matrix, $p$ is the number of effects of the statistical model (which equals the number of columns of the model matrix), and $N$ is the number of runs.
  
  When the design is organized in blocks, the calculations are different.
  - When blocking factor is treated as a fixed effect:
    
    $\text{D-efficiency} = \frac{\mathbf{{|[XZ]'[XZ]|}^{1/p}}}{N}$
    
    $\textit{A-efficiency} = \frac{p}{N \times tr\mathbf{{([XZ]'[XZ])}^{-1}}}$
    
    where $\mathbf{X}$ is the model matrix selected (ME, ME+IE, or ME+SOE) excluding the intercept, $\mathbf{Z}$ is the dummy coded matrix for the blocking factor, $\mathbf{[XZ]}$ is the concatenated full model matrix , $p$ is the total number of effects and $N$ is the number of runs.
  - When blocking factor is treated as a random effect:
    
    $\textit{D-efficiency} = \frac{\mathbf{{|X’V^{-1}X|}^{1/p}}}{N}$
    
    $\textit{A-efficiency} = \frac{p}{N \times tr\mathbf{{(X’V^{-1}X)}^{-1}}}$
    
    where $\mathbf{X}$ is the model matrix selected (ME, ME+IE, or ME+SOE) including the intercept, $\mathbf{V}$ is the variance-covariance matrix of all responses, p is the total number of effects and N is the number of runs. Here, $\mathbf{V}$ is adapted based on the intrablock correlation setting.
  It is important to note that the efficiency values will generally be low for three-level or mixed-level designs. This is because the efficiency is calculated in comparison to a theoretical optimal for a two level design.
  
  For a more thorough understanding of the concept of blocking, refer to Chapter 7 and 8 of Optimal design of experiments by Peter Goos, Bradley Jones [^gj].

José Núñez Ares & Peter Goos (2020) Enumeration and Multicriteria Selection of Orthogonal Minimally Aliased Response Surface Designs, Technometrics, 62:1, 21-36. ↩︎
José Núñez Ares, Eric Schoen, and Peter Goos. Orthogonal Minimally Aliased Response Surface Designs for Three-Level Quantitative Factors and Two-Level Categorical Factors. Statistica Sinica.-Taiwan 33.1 (2023): 107-126. ↩︎

Docs: Interactive graphical comparison

Mon, 01 Jan 0001 00:00:00 +0000

Select a subset of designs and the most important statistical quality indicators for you to access the interactive graphical comparison capabilities of our software.

The interactive graphical comparison is located in the middle section of the page. When the user selects the quality indicators that are most important for their problem, the software will provide a graphical analysis of the designs selected in the table at the top of the page.

The figure below is a screenshot of the graphical comparison tool. On the left side there is a menu with different tabs that contain the quality indicators. On the right side, a graphical analysis of the designs regarding the selected quality indicators is displayed.

Depending on how many indicators are selected, a different graph is displayed:

If the user selects 2 indicators, then a scatter plot is generated.
If the user selects 3 indicators, then a ternary plot is displayed.
When the number of selected indicators is more than 3, then the software displays a parallel coordinate plot.

The list of indicators that can be selected are:

Morphology
- Center points
- Degrees of freedom for pure error estimation
- Replicates
- Number of runs
Powers
- Power to detect a main effect
- Power to detect an interaction effect
- Power to detect a quadratic effect
Projection properties
- PEC(x,0)(ME+SOE)
- PEC(0,x)(ME+SOE)
Quality of estimation
- D- and A-efficiencies for a main effects model: D(ME), A(ME)
- D- and A-efficiencies for a main and interaction effects model: D(IE), A(IE)
- D- and A-efficiencies for a main and second-order effects model: D(SOE), A(SOE)
- Maximum 4th order correlation
Quality of prediction
- G-efficiency and prediction variances for a main effects model
- G-efficiency and prediction variances for a main and interaction effects model
- G-efficiency and prediction variances for a main and second-order effects model

The scatter plot

As all indicators are numerical, it is straightforward to plot them in a 2D scatter plot. The software does plot some points in red color: those that are pareto efficient points. When the mouse pointer stands over one of the points in the graph, the software will inform about which design it is.

The ternary plot

A ternary plot is a natural way to perform a multi-criteria comparison of a set of designs characterized by three numerical attributes. To this end,

on each vertex of the triangle one of the selected indicators is placed, and
the selected designs correspond to the colored areas in the ternary plot.

A point in the triangle corresponds to a set of weights for the three indicators. In the figure above, the left corner point corresponds to a weight vector $(1,0,0)$, the upper vertex to $(0,1,0)$, and the right vertex to $(0,0,1)$. The figure below displays some points in the triangle and their corresponding weight coordinates. A point in the center of gravity of the triangle has weight coordinates of $(1/3,1/3,1/3)$.

The color areas indicate which design(s) perform the best for the corresponding set of weights (each weight is applied to one indicator).

The parallel coordinates plot

The parallel coordinate plot is a very useful graph to display many items characterized by many attributes. You can enlarge the graph by clicking in the button, which will open a pop-up window.

Docs: Automatic recommendation of a design

Thu, 05 Jan 2017 00:00:00 +0000

In this section you can find a detailed description of the method that we use to automatically select the best design from a small set of competing designs.

Utopia method for selecting automatically a design

The starting point for our algorithm is a set of designs $D$ which are characterized by a number $h$ of numerical attributes of interest, which are specified in the set $A$.

We define the utopia point u as the vector of attribute values of an ideal design which is given by $u := (a_1, . . . , a_h)$, where $a_i$ is the best value for the $i$th attribute in $A$ over all designs in the set $D$. Then, we select the design in $D$ which is the closest to u using the usual euclidean distance, after normalizing all attributes in $A$. For further information on the utopia method, we refer to Marler and Arora (2004) ¹

This process outputs one design, unless several designs are at the same distance from $u$. Note that the utopia method implicitly gives the same weight to all attributes in $A$ and works for any cardinality of $D$.

Example

We illustrate how the method works by means of an example.

Consider a set of four designs which are characterized by 5 numerical statistical quality parameters. The designs and the numerical attributes are displayed in Table 1.

Table 1: initial data set with 4 characterized designs
design number	number of tests	4th order correlation	power interaction	power quadratic	G-efficiency for a model with intercept, main effects and second order effects
1	16	0	0.5223	0.5223	45.45
2	17	0.514	0.8005	0.5424	40.63
3	18	0.298	0.8076	0.5424	71.11
4	20	0.167	0.8184	0.6218	66.67

For some of the attributes a higher value is desired (powers and G-efficiency). For others, a lower value is preferred (number of tests and 4th order correlation). Given this information, which we call directions of improvements, the following transformation is applied to some of the columns in Table 1. The columns that are transformed are those corresponding to statistical quality parameters for which lower values are preferred, this is, the second and third columns which correspond respectively to the number of experimental tests or runs and the 4th order correlation. The transformation consists in multiplying all values in those columns by $-1$. By performing this transformation, all attributes become the bigger the better. The maximum value for each column is highlighted. Table 2 shows the data of Table 1 after applying this transformation.

Table 2: data after applying a sign transformation
design number	Number of tests	4th order correlation	power interaction	power quadratic	G-efficiency for a model with intercept, main effects and second order effects
1	-16	0	0.5223	0.5223	45.45
2	-17	-0.514	0.8005	0.5424	40.63
3	-18	-0.298	0.8076	0.5424	71.11
4	-20	-0.167	0.8184	0.6218	66.67

We then proceed to normalize the data on each column so that the minimum value becomes 0 and the maximum becomes 1. The results after normalization are displayed in Table 3. The maximum value for each column is highlighted.

Table 3: data after normalization
design number	Number of tests	4th order correlation	power interaction	power quadratic	G-efficiency for a model with intercept, main effects and second order effects
1	1	1	0	0	0.1581
2	0.75	0	0.9395	0.202	0
3	0.5	0.4202	0.9635	0.202	1
4	0	0.6751	1	1	0.8543

Next, we obtain the utopia point. It is clear that, after normalization, the utopia point has all its coordinates equal to 1.

The final calculation consists of obtaining the Euclidean distance of each design to the utopia point, which is displayed in Table 4.

Table 4: Euclidean distances to the utopia point
Design	Distance
1	1.6458
2	1.6441
3	1.1065
4	1.0615

For this example, Design 4 is the recommended one, as it is the closest to the utopia point.

We can apply the utopia selection also to a subset of the numerical attributes. Say that we mostly care about the number of tests, the 4th order correlation and the power to detect quadratic effects. Then, we can calculate the distances to the utopia point for these three attributes, obtaining the results in Table 5. In this case, Design 1 would be the recommended design, which makes sense as it is the best design in what respects the number of tests and the 4th order correlation.

Table 5: Euclidean distances to the utopia point when considering a subset of the design characteristics
Design	Distance
1	1.0000
2	1.3036
3	1.1059
4	1.0515

Marler, R., Arora , J. Survey of multi-objective optimization methods for engineering. Struct Multidisc Optim 26, 369–395 (2004). https://doi.org/10.1007/s00158-003-0368-6 ↩︎

Docs: Design size, powers and aliasing

Thu, 05 Jan 2017 00:00:00 +0000

Run size

Use the following slider to set a range for minimum and maximum run size for your design of interest. A good advice is to add a couple of runs above and below your target run size to explore the full potential of our database

Power

Use the following slider to specify the minimum power to detect one main effect, one interaction effect or one quadratic effect.
- The base model contains the intercept and all main effects.
- The powers are calculated using a signal-to-noise ratio (SNR) of 1 and a significance level $\alpha=0.05$.
- It is possible to check the powers for different SNR values and thresholds of significance at a later stage for the final chosen design.
- Technical considerations (click to unfold)
  The power to detect an effect is primarily influenced by three things; the signal-to-noise ratio, reliability of the sample size and significance threshold. We will explain all three in this text.
  
  The null hypothesis ($H_o$) is that a certain effect $\beta_i = 0$ and that the following term
  
  $F_o = \frac{\beta_{i}^{2}}{\hat{\sigma}^2\mathbf{(X’X)^{-1}_{ii}}}$
  
  follows a central F-distribution with $1$ and $n-p$ degrees of freedom where $n$ is the number of runs and $p$ is the number of parameters in the model matrix $\mathbf{X}$ including the intercept. In the expression, $\hat{\sigma}^2$ is the estimate of the mean square error from the model and $\hat{\beta}_i$ is the estimate of the size of an effect $i$.
  
  Since we are calculating power for an active effect ($\beta_i \neq 0$), in this case, the true $F$ has a non-central F-distribution, where the non-centrality parameter is calculated as follows:
  
  $\lambda = \frac{(\beta_{i}^{*})^{2}}{\sigma^{2}\mathbf{({X}‘X)^{-1} _{ii}}} = \frac{(SNR)^{2}}{\mathbf{({X}‘X)^{-1} _{ii}}}$ .
  
  Here, $\beta_{i}^{*}$ is the true size of the effect i and $\sigma^2$ is the true value of the standard error. The ratio of the true effect size and true standard error is often referred to as signal-to noise ratio (SNR). For design characterization, the default value for SNR is set to 1.
  
  Next, a significance threshold ($\alpha$) needs to be set. This is the p-value at which a significance conclusion will be drawn. For design characterization, $\alpha$ is set at 0.05.
  
  Summarized steps to calculate power for each effect:
  1. The preset value of SNR (=1) and $\mathbf{(X’X)^{-1}_{ii}}$ is used to calculate the non-centrality parameter $\lambda$.
  2. Calculate ‘Critical F-value’ from a central distribution F-table with $\alpha=0.05$, with $1$ and $n - p$ degrees of freedom.
  3. Obtain the type II error using non central F-distribution(Critical F-value, $1$ , $n - p$, $\lambda)$. The type II error is the probability of accepting the null hypothesis, when the alternative hypothesis is true. The type II error is denoted as $\beta$ (not to be mistaken with the effect size in a statistical model).
    
    $\textit{Power} = 1 - \beta$
  Note: When random block effects are included, the term $\mathbf{(X’V^{-1}X)^{-1}_{ii}}$ is used in step 1.
  
  To determine the power for:
  - Single main effect: the model matrix $\mathbf{X}$ consists of the intercept and all the main effects columns. The minimum power attained for any of the main effect is recorded.
  - Single interaction effect: the model matrix consists of the intercept and all main effects columns and the one interaction effect column. For all designs, all individual interaction effects are tested separately in a base model that includes all main effects and the minimum power attained for any of the interaction effect is recorded. The user can use the slider to set the minimum power requirements to detect a single interaction effect in a base model with all main effects.
  - Single quadratic effect: the model matrix consists of the intercept and all main effects columns and the one quadratic effect column. For all designs, all individual quadratic effects are tested separately in a base model that includes all main effects and the minimum power attained for any of the quadratic effect is recorded. The user can use the slider to set the minimum power requirements to detect a single quadratic effect in a base model with all main effects.
  When fixed block effects are considered, the model matrix will also include dummy coded block effect columns.

Aliasing

Use the following slider to fix the maximum absolute value of the aliasing/correlation between any two second-order effect. The slider ranges from 0 to 1.

Docs: Morphology

Thu, 05 Jan 2017 00:00:00 +0000

The content on this tab refer to qualities of the design matrix. For example, there is information on the factors, the number of runs, whether the design is foldover, etc.

As in the previous tab, when you place the pointer of your mouse over the text on the Properties column, you will get an explanation.

Docs: Aliasing

Thu, 05 Jan 2017 00:00:00 +0000

This tab contains a colormap on correlations with individual absolute correlations between the effects in a full second-order effects model, together with some averages and maximum values that appear on it.

As in the previous tab, when you place the pointer of your mouse over the text on the Properties column, you will get an explanation.

Docs: Catalog searches

Thu, 05 Jan 2017 00:00:00 +0000

This table shows which filterings of the catalog you saved for your research questions. The saved filterings make it easy to continue your search for the best designs.

Docs: Projection properties

Thu, 05 Jan 2017 00:00:00 +0000

Often, only a limited number of the factors in an experiment turn out to have significant effects. Ideally, the design allows you to obtain a good quality second-order model involving these factors. Such a model includes all their main effects, interaction effects and, in case of 3-level factors, quadratic effects.

Before doing the experiment, you don’t know exactly which of the factors will have significant effects. However, you may have some idea about the maximum number of factors involved.

In general, if you have $p$ factors and believe that at most $q$ of them can show significant effects, there are $\binom{p}{q}$ projections, and a design that performs well on average over all the $q$-factor projections is preferred.
Example: suppose that your design includes 10 factors, and you believe that at most 5 of them can show significant effects. The design has $\binom{10}{5}=252$ different subsets of 5 factors. In other words, there are 252 different projections of the design into 5 factors. A design with good average estimation quality over all 252 projections would serve your turn.

This tab allows you to specify:

The maximum number of factors that you believe to have significant effects
- The controls allow specification for the maximum number in models with only three-level factors, in models with only two-level factors and in models with a combination of three-level and two-level factors.
The minimum values for two criteria to quantify how efficiently the design quantifies a second-order model with this number of factors.
- Both the sliders and the fill-out boxes allow you to specify minimum values for the average D-efficiency, the average A-efficiency, or both.
- Both of these criteria address the statistical uncertainty in the model coefficients. This uncertainty can be expressed as the variance of the individual coefficients and the correlation of pairs of coefficients. The variance of a coefficient measures how precise the coefficient itself is estimated. The correlation among two coefficients measures the amount of cross-talk between them. If two coefficients are highly correlated, you cannot be sure which of the corresponding effects contributes to either coefficient. If they are uncorrelated, the interpretation of the coefficients is unambiguous.
- Technical considerations (click to unfold)
  - A design’s D-efficiency for a particular model addresses the variance of the model coefficients as well as the correlation between these coefficients. Maximizing the D-efficiency simultaneously minimizes the uncertainty in the coefficients themselves and the cross-talk between coefficients.
  - The A-efficiency of a design for a particular model solely addresses the variance of the model coefficients. Maximizing the A-efficiency minimizes the uncertainty in the coefficients, but leaves the amount of cross-talk between coefficients unaddressed.
  - Both kinds of efficiency can attain values between 0% and 100%. An average of 0% means that the model cannot be calculated for any of the projections. An average of 100% means that the models for the projections all have the best possible efficiency for all the projections. If the average is in between, the models don’t all have the best efficiency but neither do they all have the worst possible efficiency.
  - The technical definition of the D-efficiency of a model with p parameters (including intercept) based on a design with N runs is $$D = 100 \cdot \frac{|\mathbf{X}^T\mathbf{X}|^{1/p}}{N}, $$
    where |.| denotes the determinant, and $\mathbf{X}$ is an $N \times p$ model matrix with intercept, and further independent variables corresponding to some or all main effects, interaction effects and quadratic effects. Each effect in the matrix is normalized to a length of $\sqrt{N}$.
  - The technical definition of the A-efficiency of a model with p parameters based on a design with N runs is $$A = 100 \cdot \frac{p}{N \cdot tr[(\mathbf{X}^T\mathbf{X})^{-1}]},$$ where $tr[.]$ denotes the trace of a matrix, $(.)^{-1}$ denotes the inverse of a matrix and $\mathbf{X}$ is defined above.

The two sets of controls allow you to specify projection estimability requirements for two types of subsets of effects at the same time.

Docs: Advanced options

Thu, 05 Jan 2017 00:00:00 +0000

Center points, pure error and uniform precision designs

A center point is a design point where all the three-level factors are set at their middle level. The slider designated ‘Center Points’ allows you to specify a range of numbers of such points.
The slider designated DF for pure error estimation allows you to specify the minimum number of degrees of freedom for a model-free estimate of the random error. This number is determined as the number of runs minus the number of distinct points of the design. If you want to base the statistical tests solely on the model-free estimate, a minimum of 3 DF for pure error estimation is recommended.
- If the design includes only three-level factors, including two or more center points allows for model-free estimation of the random error, also called pure error estimation. The corresponding number of degrees of freedom is one less than the number of center points. If the design includes additional two-level factors, the number of center points is the total over all the combinations of two-level factors. In that case, including more than one center point may or may not allow for pure error estimation.
There are two check boxes to specify conditions on the number of runs at the middle level of the three-level factors. Checking the box Main and quadratic effects ensures that the number of runs at the middle level of any numerical three-level factor is the same. This ensures a roughly equal precision for the main effects and for the quadratic effects in models with just these effects.
Checking the box Interaction effects ensures that, for any pair of numerical three-level factors, the number of runs where one or both of the factors attain their middle level is the same. This ensures an equal precision for the interaction effects in all models with main effects and a single interaction effect.

Docs: Efficiencies

Thu, 05 Jan 2017 00:00:00 +0000

This tab displays the efficiencies for the models that are estimable.

As in the previous tab, when you place the pointer of your mouse over the text on the Properties column, you will get an explanation. You can change the model using the radio buttons on top.

The D-, A-, G- and I-efficiency are standard attributes of an experimental design which inform us about how well the model effects are estimated and how precise the predictions will be. All efficiencies are, in one way or another, based on the Fisher information matrix.

Design matrix and model matrix

The design matrix, denoted as $\mathbf{D}$, has $n$ rows (one per test), and $m$ columns (one per factor). Schematically, it is represented as $\mathbf{D} = \begin{pmatrix} d_{11} & d_{12} & \cdots & d_{1m} \\ d_{21} & d_{22} & \cdots & d_{2m} \\ \vdots & \vdots & \cdots & \vdots \\ d_{n1} & d_{n2} & \cdots & d_{nm} \end{pmatrix}$, where an entry $d_{ij}$ indicates the level of the $j$-th factor at the $i$-th run.

The expansion vector function, denoted as $\mathbf{f}(\mathbf{d})$ takes a vector of factor settings (this is, a row of $\mathbf{D}$), and expands that vector to its correponding model terms, which can include intercept, main effects, two-factor interaction effects, quadratic effects, etc.

The model matrix, denoted as $\mathbf{X}$, has $n$ rows (one per test), and $p$ columns (one per effect). It is built by applying the expansion vector function to each row of the design matrix.

D-efficiency and A-efficiency

Notation:

D-efficiency for a specific model: D(model)
A-efficiency for a specific model: A(Model)

Both measure how precisely the effects in a model can be estimated.

The inverse of the information matrix, $(\mathbf{X}^T\mathbf{X})^{-1}$, indicates how well the model effects can be estimated. In an orthogonal design, where the model effects can be estimated independently from each other, both the information matrix and its inverse are diagonal matrices.

For example, the diagonal entries quantify the estimation error of each one of the model effects, and its determinant gives an overall measure on how good all model effects can be estimated compared to an orthogonal design.

The D-efficiency is a number that lies between $0$ and $100$ and gives us information on the confidence ellipsoid of the estimation error of the model effects. The lower this value, the better. It is calculated from the determinant of the information matrix as follows: $D-efficiency = 100 \cdot\frac{|\mathbf{X}^T\mathbf{X}|}{n}^{1/p}$, where $|\cdot|$ denotes the determinant of a matrix, $p$ is the number of effects in the model and $n$ is the number of runs. The D-efficiency takes into account the correlation between the estimates of the model effects.

The A-efficiency is a number that lies between $0$ and $100$ and, similarly to the D-efficiency, indicates how well the model effects can be estimated overall. It is calculated using the trace of the information matrix as follows: $A-efficiency = 100 \cdot \frac{p}{n \cdot tr((\mathbf{X}^T\mathbf{X})^{-1})}$, where $tr(\cdot)$ is a matrix operator result of the sum of the diagonal entries of the matrix (matrix trace).

G-efficiency and average prediction variance

Notation:

G-efficiency for a specific model: G(model)
Average prediction variance for a specific model: Avg pred var(Model)

The variance of the predicted value at a point $\mathbf{x}$ is given by: $pv_{\mathbf{x}}= \mathbf{f}^T(\mathbf{x}) ( \mathbf{X}^T\mathbf{X} )^{-1} \mathbf{f}(\mathbf{x})\sigma_{\varepsilon}^2$, where $\mathbf{f}(\mathbf{x})$ is the model expansion of $\mathbf{x}$ over the experimental region $\chi$, and $\sigma_{\varepsilon}^2$ is the error variance.

The maximum prediction variance then equals $\max pv_{\mathbf{x}}, \mathbf{x} \in \chi$, and the average prediction variance $avp_{\chi} = \int_{\chi} pv_{\mathbf{x}} d\mathbf{x}$.

Finally, $G-efficiency = 100 \cdot \frac{p}{n \cdot avp_{\chi}}$.

Docs: Detailed comparison

Mon, 01 Jan 0001 00:00:00 +0000

Docs: Powers

Thu, 05 Jan 2017 00:00:00 +0000

We look at the designs from different angles with respect to the power calculation. Some of the powers we present here are not present in any other commercial software. Here you will find detailed information on how we calculate them.

The information on this interactive table helps assess how good a design is regarding the capacity to estimate each effect type. The table has three columns:

Model (1st column): underlying set of effects considered for the power calculation. Only models that are estimable within the design are displayed. The models may be one of the following:
- Model with a constant term (intercept)
- Model with a constant term and main effects (me)
- Model with a constant term, main effects, and two-factor interaction effects (meie)
- Model with a constant term, main effects, two-factor interaction effects, and quadratic effects (mesoe)
- Model with a constant term, main effects, and quadratic effects (meqe)
Effects (2nd column): the effect type considered. It can be one of the following:
- Main effect (main)
- Two-factor interaction effect (interaction)
- Quadratic effect (quadratic)
Value (3rd column): minimum value of the power to detect one effect of the type mentioned in the second column when the underlying model has the effects mentioned in the first column.

Try to change the values of $\alpha$ and the signal-to-noise ratio (SNR) to see how the values of the power change.

Docs: Projection properties

Thu, 05 Jan 2017 00:00:00 +0000

While a design may not allow the estimation of a full second-order effects model, it may allow the estimation of a large subset of the effects, so that it is suitable for a screening + optimization experiment at the same time. Second-order models for subsets of factors.

This page offers details on the estimation quality of second-order models based on subsets of the factors. For any subset, these models include an intercept, all the main effects, all the two-factor interactions, and all the quadratic effects of the numerical three-level factors in the subset.

The numbers of numerical three-level factors in the subsets are specified in the first column of the table.
The numbers of two-level factors are specified in the table’s second column.
The third column displays the so-called projection estimation capacity for up to three designs that are to be compared. This is the fraction of estimable models for the numbers of numerical three-level factors and two-level factors specified in the first and second columns, respectively.
The last column contains clickable buttons that produce, for each subset with the numbers of two-level and three-level factors specified in the first two columns, the average D-efficiencies and A-efficiencies. Further characteristics are the averages of (1) the maximum prediction variance and (2) the average prediction variance over 500 randomly chosen combinations of two-level and three-level factors.
The combinations of the two-level factors are randomly chosen from the full factorial design in the specified number of two-level factors. The combinations of the three-level factors are randomly chosen from between the low and high levels.
As the maximum prediction variance is often located at the edges and corners, a few of such points are added to the 500 random points.

Docs: Model effects

Thu, 05 Jan 2017 00:00:00 +0000

When a certain model is estimable, we can obtain statistical quality indicators that gives us information on the power to estimate each effect in the model, the variance inflation factor of each estimate in the model, or the relative error of estimation of each individual effect.

The power to detect an active effect.

Calculated for different values of Signal-to-noise ratio and significance level ($\alpha$). For detailed information on how the power is calculated, please click here.

Variance inflation factor

Variance inflation factor (VIF) is a measure of how much the variance of the estimate of an individual effect is inflated due to aliasing or multicollinearity with other effects. The expression for VIF is given as:

$VIF = \frac{1}{1-R^2}$

where $R^2$ is the coefficient of determination of a model where the dependent variable is the selected effect and the independent variables are all the other terms in the model selected. A VIF value of 1 ($R^2 = 0$) indicates that there is no aliasing between the selected effect and all other effects included in the model. Ideally this number should be small ($<10$).

Relative error of estimation

The Relative error of estimation (REE) is the standard error of the estimate of an effect relative to the standard error of the model. This is given by the expression:

$REE = \sqrt{(\mathbf{X’X})^{-1}_{ii}}$ ,

where $\mathbf{X}$ is the selected model matrix and $(\mathbf{X’X})^{-1}_{ii}$ is the $i^{th}$ diagonal element of the variance covariance matrix $(\mathbf{X’X})^{-1}$ . The actual standard error of an estimate of an effect can be calculated by multiplying this expression with the standard error from the model.

All the three characterizations (POWER, VIF, REE) can be calculated for the following models:

ME – All main effects model
MEIE – All main effects + all two-factor interactions model
MESOE – All main effects + all second order effects (two-factor interactions and quadratic effects) model.
MEQE – Main effects + all quadratic effects model.

Additional notes:

Some model options may not appear for the chosen design. This happens when the said model cannot be fit with the design selected.

The powers and REE for the blocking effect is only displayed when blocking option is selected (Under – ‘Define your experiment’) and the intrablock correlation coefficient is set to Extreme (For more information, refer to ‘power page’).

The VIF and REE calculations are independent from the specified value of Signal-to-noise ratio and significance level $\alpha$.