Effex app documentation – Comparison

Docs: Interactive graphical comparison

Mon, 01 Jan 0001 00:00:00 +0000

Select a subset of designs and the most important statistical quality indicators for you to access the interactive graphical comparison capabilities of our software.

The interactive graphical comparison is located in the middle section of the page. When the user selects the quality indicators that are most important for their problem, the software will provide a graphical analysis of the designs selected in the table at the top of the page.

The figure below is a screenshot of the graphical comparison tool. On the left side there is a menu with different tabs that contain the quality indicators. On the right side, a graphical analysis of the designs regarding the selected quality indicators is displayed.

Depending on how many indicators are selected, a different graph is displayed:

If the user selects 2 indicators, then a scatter plot is generated.
If the user selects 3 indicators, then a ternary plot is displayed.
When the number of selected indicators is more than 3, then the software displays a parallel coordinate plot.

The list of indicators that can be selected are:

Morphology
- Center points
- Degrees of freedom for pure error estimation
- Replicates
- Number of runs
Powers
- Power to detect a main effect
- Power to detect an interaction effect
- Power to detect a quadratic effect
Projection properties
- PEC(x,0)(ME+SOE)
- PEC(0,x)(ME+SOE)
Quality of estimation
- D- and A-efficiencies for a main effects model: D(ME), A(ME)
- D- and A-efficiencies for a main and interaction effects model: D(IE), A(IE)
- D- and A-efficiencies for a main and second-order effects model: D(SOE), A(SOE)
- Maximum 4th order correlation
Quality of prediction
- G-efficiency and prediction variances for a main effects model
- G-efficiency and prediction variances for a main and interaction effects model
- G-efficiency and prediction variances for a main and second-order effects model

The scatter plot

As all indicators are numerical, it is straightforward to plot them in a 2D scatter plot. The software does plot some points in red color: those that are pareto efficient points. When the mouse pointer stands over one of the points in the graph, the software will inform about which design it is.

The ternary plot

A ternary plot is a natural way to perform a multi-criteria comparison of a set of designs characterized by three numerical attributes. To this end,

on each vertex of the triangle one of the selected indicators is placed, and
the selected designs correspond to the colored areas in the ternary plot.

A point in the triangle corresponds to a set of weights for the three indicators. In the figure above, the left corner point corresponds to a weight vector $(1,0,0)$, the upper vertex to $(0,1,0)$, and the right vertex to $(0,0,1)$. The figure below displays some points in the triangle and their corresponding weight coordinates. A point in the center of gravity of the triangle has weight coordinates of $(1/3,1/3,1/3)$.

The color areas indicate which design(s) perform the best for the corresponding set of weights (each weight is applied to one indicator).

The parallel coordinates plot

The parallel coordinate plot is a very useful graph to display many items characterized by many attributes. You can enlarge the graph by clicking in the button, which will open a pop-up window.

Docs: Automatic recommendation of a design

Thu, 05 Jan 2017 00:00:00 +0000

In this section you can find a detailed description of the method that we use to automatically select the best design from a small set of competing designs.

Utopia method for selecting automatically a design

The starting point for our algorithm is a set of designs $D$ which are characterized by a number $h$ of numerical attributes of interest, which are specified in the set $A$.

We define the utopia point u as the vector of attribute values of an ideal design which is given by $u := (a_1, . . . , a_h)$, where $a_i$ is the best value for the $i$th attribute in $A$ over all designs in the set $D$. Then, we select the design in $D$ which is the closest to u using the usual euclidean distance, after normalizing all attributes in $A$. For further information on the utopia method, we refer to Marler and Arora (2004) ¹

This process outputs one design, unless several designs are at the same distance from $u$. Note that the utopia method implicitly gives the same weight to all attributes in $A$ and works for any cardinality of $D$.

Example

We illustrate how the method works by means of an example.

Consider a set of four designs which are characterized by 5 numerical statistical quality parameters. The designs and the numerical attributes are displayed in Table 1.

Table 1: initial data set with 4 characterized designs
design number	number of tests	4th order correlation	power interaction	power quadratic	G-efficiency for a model with intercept, main effects and second order effects
1	16	0	0.5223	0.5223	45.45
2	17	0.514	0.8005	0.5424	40.63
3	18	0.298	0.8076	0.5424	71.11
4	20	0.167	0.8184	0.6218	66.67

For some of the attributes a higher value is desired (powers and G-efficiency). For others, a lower value is preferred (number of tests and 4th order correlation). Given this information, which we call directions of improvements, the following transformation is applied to some of the columns in Table 1. The columns that are transformed are those corresponding to statistical quality parameters for which lower values are preferred, this is, the second and third columns which correspond respectively to the number of experimental tests or runs and the 4th order correlation. The transformation consists in multiplying all values in those columns by $-1$. By performing this transformation, all attributes become the bigger the better. The maximum value for each column is highlighted. Table 2 shows the data of Table 1 after applying this transformation.

Table 2: data after applying a sign transformation
design number	Number of tests	4th order correlation	power interaction	power quadratic	G-efficiency for a model with intercept, main effects and second order effects
1	-16	0	0.5223	0.5223	45.45
2	-17	-0.514	0.8005	0.5424	40.63
3	-18	-0.298	0.8076	0.5424	71.11
4	-20	-0.167	0.8184	0.6218	66.67

We then proceed to normalize the data on each column so that the minimum value becomes 0 and the maximum becomes 1. The results after normalization are displayed in Table 3. The maximum value for each column is highlighted.

Table 3: data after normalization
design number	Number of tests	4th order correlation	power interaction	power quadratic	G-efficiency for a model with intercept, main effects and second order effects
1	1	1	0	0	0.1581
2	0.75	0	0.9395	0.202	0
3	0.5	0.4202	0.9635	0.202	1
4	0	0.6751	1	1	0.8543

Next, we obtain the utopia point. It is clear that, after normalization, the utopia point has all its coordinates equal to 1.

The final calculation consists of obtaining the Euclidean distance of each design to the utopia point, which is displayed in Table 4.

Table 4: Euclidean distances to the utopia point
Design	Distance
1	1.6458
2	1.6441
3	1.1065
4	1.0615

For this example, Design 4 is the recommended one, as it is the closest to the utopia point.

We can apply the utopia selection also to a subset of the numerical attributes. Say that we mostly care about the number of tests, the 4th order correlation and the power to detect quadratic effects. Then, we can calculate the distances to the utopia point for these three attributes, obtaining the results in Table 5. In this case, Design 1 would be the recommended design, which makes sense as it is the best design in what respects the number of tests and the 4th order correlation.

Table 5: Euclidean distances to the utopia point when considering a subset of the design characteristics
Design	Distance
1	1.0000
2	1.3036
3	1.1059
4	1.0515

Marler, R., Arora , J. Survey of multi-objective optimization methods for engineering. Struct Multidisc Optim 26, 369–395 (2004). https://doi.org/10.1007/s00158-003-0368-6 ↩︎

Docs: Detailed comparison

Mon, 01 Jan 0001 00:00:00 +0000