# 1. Experimental Modeling (Identification)

## Vision: Automatic Modeling

## Local Model Network

### Introduction

Model output is calculated by summing up the contributions of all M local models (LMs):

### Advantages

- Separate input spaces for validity functions (rule premises) z and local models (rule consequents) x.
- Local estimation of local model parameters much more robust → Regularization.
- Local models can be of arbitrary type:

- linear: number of parameters scale linearly with dim(x)

- quadratic: optimization,

- higher order polynomials: subset selection or ridge regression required,

- from first principles: prior knowledge effectively utilized. - Exact shape of validity function not relevant.
- Efficient greedy learning strategies are available.

### Drawbacks

- Undesired interpolation side effect.
- Undesired normalization side effects.
- Suboptimal.

### Separate Input Spaces for Validity Functions z and Local Models x

- Additional flexibility.
- Better interpretation.
- z can be seen as scheduling variables (often low-dimensional).
- x often is high-dimensional, particularly for dynamic systems.

### Interpretation as Takagi-Sugeno Fuzzy System

### Partion of Unity

### Two Ways to Achieve This

1.*Normalization*based on Gaussians:

- Defuzzification
- Undesirable side effects

2.

*Hierarchy*based on sigmoids

- Binary tree
- Knots weighted with Psi and 1-Psi

### Algorithm 1: Local Linear Model Tree (LOLIMOT)

**Properties:**

*Partitioning:*Axes-orthogonal.*Structure:*Flat (parallel).*Splitting functions:*Normalized Gaussian functions.*Splitting method:*Heuristically, without nonlinear optimization.

### LOLIMOT Construction Algorithm

- Incremental (growing) algorithm: Adds one LM in each iteration.
- Split of the locally worst LM.
- Test of all splitting dimensions and selection of the best alternative.
- Local least squares estimation of the LM parameters.
- Use normalized Gaussian validity functions.

### Demonstration Example (Hyperbola)

### Algorithm 2: Hierarchical Local Model Tree (HILOMOT)

**Properties:**

*Partitioning:*Axes-oblique.*Structure:*Hierarchical.*Splitting functions:*Sigmoid functions.*Splitting method:*Nonlinear optimization of sigmoid parameters.

### HILOMOT Optimization

- Nested approach.
- Separable nonlinear least squares.
- Outer loop: Gradient-based nonlinear optimization of split.
- Inner loop: One-shot least squares (LS) optimization of local model parameters.

### Enhancements

- Regularization of LS optimization (ridge regression).
- Subset selection instead of LS can determine local structure.
- Constraint nonlinear optimization of the splits guarantees for all local models .
- Analytical gradient calculation speeds up optimization by factor
*dim*().__u__ - Derivative of inverse matrix (LS estimation) necessary.
- Automatic sophisticated smoothness adjustment of sigmoid splitting functions.

### Separable Nonlinear Least Squares

### HILOMOT: Extension and Modification of LOLIMOT

- Incremental (growing) algorithm: Adds one LM in each iteration → like LOLIMOT.
- Split of the locally worst LM → like LOLIMOT.
- Optimize splitting position and angle by nonlinear optimization → new.
- Automatic smoothness adjustment algorithm → new.
- Local least squares estimation of the LM parameters → like LOLIMOT.
- Use sigmoid validity functions → different to LOLIMOT.
- Build up a hierarchical model structure → different to LOLIMOT.

### Demonstration Example (Hyperbola)

### Properties of LOLIMOT

- Axes-orthogonal splits.
- Flat structure can be computed in parallel.
- Strongly suboptimal in high dimensions.
- Faster training.
- Easier to understand and interpret.
- Normalization side effects.

### Properties of HILOMOT

- Axes-oblique splits.
- Hierarchical structure fully decouples local models.
- Well suited for high dimensions.
- Analytical gradient calculation for split optimization.
- Superior model accuracy.
- No normalization numerator (defuzzification). → No reactivation effects. Hierarchy automatically guarantee a ‘partition of unity’.
- Sigmoids are easier to approximate than Gaussians for microcontroller implementation.

### Summary: Advantages of Local Model Networks

- Incremental → all models from simple to complex are constructed.
- Local estimation:

- All local model are decoupled.

- Extremely fast.

- Regularization effect → almost no overfitting. - Local model are linear in their parameters:

- Global optimum is found.

- Robust & mature algorithms for least squares estimation are utilized.

- Structure selection technique can be applied.

- Adaptive models → recursive algorithms can be applied. - Local model can be of any linearly parameterized type: constant, linear, quadratic, ...
- Different input spaces for validity functions (rule premises) and local models (rule consequents) → new approaches to dimensionality reduction.
- Complex problem is divided into smaller sub-problems (local models) → divide & conquer.
- Many concepts from the linear world can be transferred to the nonlinear world → particularly important for dynamic models.

### LMNtool

- MATLAB toolbox.
- Object-oriented programming.
- Local linear and quadratic models.
- LOLIMOT (axes-orthogonal partitions).
- HILOMOT (axes-oblique partitions).
- Model complexity chosen according to corrected Akaike information criterion.
- Analytical gradient calculation for high training speed
- Reproducible results
- All fiddle parameters can keep their default values. No expertise required.
- Trivial usage: